In its continuing efforts to keep the public informed about the ongoing admissions litigation, the University of Michigan makes these transcripts of the trial proceedings in Grutter v Bollinger, et al., Civil Action No. 97-75928 (E.D. Mich.), available to the University community and general public. As is often the case with transcription, some words or phrases may be misspelled or simply incorrect. The University makes no representation as to the accuracy of the transcripts.
1
1 UNITED STATES OF AMERICA
2 IN THE UNITED STATES DISTRICT COURT
3 FOR THE EASTERN DISTRICT OF MICHIGAN
4 SOUTHERN DIVISION
5 BARBARA GRUTTER,
6 for herself and all others
7 similarly situated,
8 Plaintiff,
9 -vs- Case Number:
10 97-CV-75928
11 LEE BOLLINGER, JEFFREY LEHMAN,
12 DENNIS SHIELDS, and REGENTS OF
13 THE UNIVERSITY OF MICHIGAN,
14 Defendants.
15 -and-
16 KIMBERLY JAMES, et. al.,
17 Intervening Defendants.
18 ______________________________________/ VOLUME IV
19 BENCH TRIAL
BEFORE THE HONORABLE BERNARD A. FRIEDMAN
20 United States District Judge
238 U.S. Courthouse & Federal Building
21 231 Lafayette Boulevard West
Detroit, Michigan 48226
22 Friday, January 19, 2001
23 APPEARANCES:
24 FOR PLAINTIFF: Kirk O. Kolbo, Esq.
25 R. Lawrence Purdy, Esq.
2
1 APPEARANCES (CONTINUING)
2 FOR DEFENDANTS: John Payton, Esq.
3 Craig Goldblatt, Esq.
4 Stuart Delery, Esq.
5 On behalf of the Defendants
6 Bollinger, et. al.
7
8 George B. Washington, Esq..
9 Miranda K.S. Massie, Esq.
10 On behalf of Intervening Defendants.
11
12 COURT REPORTER: MARY F. WISNESKI, CSR-0231
13 Official Court Reporter
14
15
16 Proceedings recorded by mechanical stenography.
17 Transcript produced by computer-assisted
18 transcription
19
20
21
22
23
24
25
3
1 I N D E X
2 WITNESS PAGE
3 STEPHEN W. RAUDENBUSH
4 Direct Examination by Mr. Delery 5
5 Cross-Examination by Ms. Massie 118
6 Cross-Examination by Mr. Kolbo 121
7 Redirect Examination by Mr. Delery 160
7 DENNIS SHIELDS
8 Direct Examination by Mr. Payton 162
9 Cross-Examination by Mr. Purdy 193
10 Redirect Examination by Mr. Payton 215
11 Recross-Examination by Mr. Purdy 218
12 E X H I B I T S
13
14 NUMBER IDENTIFICATION ADMITTED
15 145 Expert Witness Report of S. Raudenbush 12
16 146-150 Supp. Expert Witness Rep. of S. Raudenbush 12
17 151 Raudenbush Curriculum Vitae 9
18 184-194 Charts of S. Raudenbush 108
19 5 Gospel According to Dennis 188
20
21
22
23
24
25
4
1/19/01 - BENCH TRIAL - VOLUME IV
1 Detroit, Michigan
2 January 19, 2001
3 * * *
4 THE COURT: Good morning, everyone. On the
5 motions, I have nothing else on the agenda this case, why
6 don't we start the case and when we take it a break
7 sometime we'll argue those motions.
8 MS. MASSIE: That sounds great.
9 THE COURT: Is that good for everybody? I'm all
10 prepared, but I just don't want to waste your time this
11 morning. I know you have a witness. This is yours?
12 MR. DELERY: Yes. Good morning, Your Honor,
13 Stewart Delery, Your Honor, again for the university and
14 the individual defendants.
15 THE COURT: How are you.
16 MR. DELERY: If you're ready to proceed.
17 THE COURT: I'm ready. If you're ready, I'm
18 ready. We call.
19 MR. DELERY: We call as our next witness, Stephen
20 Raudenbush.
21 THE COURT: For evidence?
22 MR. DELERY: Thank you, Your Honor.
23 S T E P H E N W. R A U D E N B U S H
24 was called as a witness and after having been
25 sworn was examined and testified as follows:
5
1/19/01 - BENCH TRIAL - VOLUME IV
1 DIRECT EXAMINATION
2 BY MR. DELERY:
3 Q. Could you please state your name and address for the
4 record.
5 A. Stephen W. Raudenbush, 7 Harvard Place, Ann Arbor,
6 Michigan.
7 Q. And where do you work?
8 A. I work at the University of Michigan.
9 Q. What's your job there?
10 A. I'm a professor in the School of Education and the
11 Department of Statistics, and I also have a joint
12 appointment as a Senior Research Scientific at the Survey
13 Research Center.
14 Q. How long have you been at the University of Michigan?
15 A. I've been at Michigan since January 1 of 1998.
16 Q. And where were you before that?
17 A. For fourteen years before that I was at Michigan State
18 University.
19 Q. Well, Professor Raudenbush, could you please, please
20 briefly describe your education, or educational background
21 for the Court.
22 A. Sure. I received my bachelor's degree from Harvard
23 College in 1968 and my doctoral degree from Harvard
24 University in 1984.
25 Q. Has your work at the University of Michigan and before
6
1/19/01 - BENCH TRIAL - VOLUME IV
1 that at Michigan State focused on any particular areas?
2 A. Yes, it has. It's, primarily my work is involved
3 applications of statistics in education, studying student
4 learning, studying student transitions into college,
5 studying how schools and classrooms effect academic
6 achievement. And also looking at other aspects of human
7 development.
8 Q. Okay. And have you published in these fields?
9 A. Yes, I have.
10 Q. About how many publications have you had?
11 A. Well, I guess if you count the second edition of our
12 book on Hierharchical Linear Models, if you count the
13 second edition of our book on Hierarchical Linear Models.
14 THE COURT: Do you want it spelled?
15 (Whereupon an off-the-record
16 discussion was had.)
17 A. H-i-e-r-h-a-r-c-h-i-c-a-l. Okay. There would be, if
18 you count that one, there would be four books and quite a
19 large number of referee journal articles and book chapters
20 that I've published over the years. I'm not sure exactly
21 how many but I publish about four to six articles and
22 chapters a year.
23 Q. Okay. This may be a relative question, but are any of
24 those publications particularly widely known?
25 A. Well, the book I mentioned, I won't mention the title
7
1/19/01 - BENCH TRIAL - VOLUME IV
1 again, has become very, very widely used in education
2 because it deals with the problem of students being nested
3 within classrooms, classrooms within schools. Those kinds
4 of problems become very widely used. And other aspects of
5 social science where we have people in neighborhoods, or we
6 have small groupings of people, basically, which has some
7 relevance to this case.
8 Q. Are you a member of any professional organizations?
9 A. I am. I'm a member of the American Statistical
10 Association, the American Educational Research Association.
11 I'm a member of the National Academy of Education.
12 Q. What's the National Academy of Education?
13 A. Well, the National Academy of Education is an honorary
14 association limited to 125 people in the United States who
15 are involved in education and educational research.
16 Q. Have you held any editorial positions for journals or
17 other publications in your field?
18 A. I have. I've been an Associate Editor of the Journal
19 of Educational and Behavioral Statistics for quite a large
20 number of years. I was the Chair of the Management
21 Committee of that journal for six years. I have served on
22 the Publications Management Committee of the American
23 Statistical Association. I'm also the Associate Editor for
24 the American Journal of Sociology, Educational Evaluation
25 and Policy Analysis and actually several other journals. I
8
1/19/01 - BENCH TRIAL - VOLUME IV
1 won't list them all.
2 Q. Okay. Have you received any teaching or other honors
3 in your field?
4 A. I have. I received, while I was at Michigan State, I
5 received three teaching awards. I've also received several
6 awards for outstanding publications in education and
7 sociology.
8 Q. Okay. Are there any awards or honors that you think
9 are particularly significant?
10 A. I think perhaps the one that I'm, maybe most proud of
11 is that in 1993 I received the Early Career Award for the
12 American, from the American Educational Research
13 Association, which is a very large group of educators and
14 educational researchers around the country.
15 Q. What about national panels or symposia? Have you
16 participated in any of those?
17 A. Yes. In the last, within the last three years, I
18 served on the National Academy of Sciences' panel on the
19 assessment of children in conjunction, basically testing,
20 in conjunction with the Title I Program, which is a
21 compensatory education program. I also served on the
22 National Academy of Science panel on early childhood
23 science, which has just distributed a new book on childhood
24 science with implications for policy and practice.
25 Q. Okay. Professor Raudenbush, I'd like to ask you to
9
1/19/01 - BENCH TRIAL - VOLUME IV
1 look at Exhibit 151, which is, I think in binder six, Your
2 Honor.
3 A. Okay. I see it.
4 Q. Okay. Is that a current copy of your CV?
5 A. It does indeed appear to be that, yes, a current copy.
6 Q. And does it include a current list of your
7 publications and honors and professional experiences?
8 A. Yes, it does.
9 MR. DELERY: Your Honor, at this time, we'd offer
10 Exhibit 151 into evidence?
11 THE COURT: Received.
12 Q. Now, Professor Raudenbush, how would you come to be
13 involved in this case?
14 A. Actually, you asked me if I'd be willing to serve as
15 an expert in this case. Can you hear me?
16 Q. Yes, I can.
17 THE COURT: If anybody can't, let us know.
18 A. Yeah. I had to move this because I can't turn the
19 page.
20 THE COURT: Yeah, that's correct.
21 Q. And what was the purpose of your involvement in the
22 case?
23 A. Well, I started by looking at some of the expert
24 reports written by Professor Kinley Larntz and I then got
25 involved in looking at the database myself, in trying to
10
1/19/01 - BENCH TRIAL - VOLUME IV
1 understand some of the issues involved in this controversy.
2 Q. Okay. Were you present here in court for Dr. Larntz'
3 testimony on Wednesday?
4 A. Yes, I was.
5 Q. And you were here for the entire day for all the
6 testimony?
7 A. I was.
8 Q. And what about on Thursday morning, yesterday morning
9 when he returned?
10 A. I was here then too, yes, correct.
11 Q. Dr. Larntz at one or two points said that he was
12 responding to some criticism of his work. Do you recall
13 that?
14 A. I do.
15 Q. Were you the author of that criticism?
16 A. I'm quite sure that I was.
17 Q. And before this week, had you ever met Dr. Larntz?
18 A. No.
19 Q. Are you being compensated for your work in this case?
20 A. No, I'm not.
21 Q. And have you ever served as an expert witness before?
22 A. No, I have not.
23 Q. Have you prepared expert reports, based on your work
24 in this case?
25 A. Yes, I have.
11
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. Okay. If you could look in the same binder there,
2 binder six, I'd like for you to look at Exhibit 145 to 150
3 and tell the court whether those are the expert reports
4 that you submitted in this case?
5 A. Yes, these are, these are the expert reports.
6 Q. What information did you consider in preparing your
7 expert reports?
8 A. Well, I read the law school admission policy, which
9 was dated 1992. And I examined data from the database made
10 available by the law school.
11 Q. Did you review the expert reports of Dr. Larntz?
12 A. Yes, I read also each, each expert report that Dr.
13 Larntz wrote.
14 Q. Okay. And what about any deposition testimony in the
15 case, did you review any of that?
16 A. Yes. I read Dr. Larntz' deposition. Of course I read
17 my own.
18 Q. Did anybody help you with your work in this matter?
19 A. Yes. Julia Smith, who was at that time a
20 post-doctoral fellow at Michigan, helped me. She's now an
21 assistant professor. And in certain aspects of the work
22 the, basically the diversity of context for learning part,
23 I received some help from two graduate students at the
24 University of Michigan.
25 MR. DELERY: Your Honor, at this point we'd offer
12
1/19/01 - BENCH TRIAL - VOLUME IV
1 Exhibit 145 through 150 into evidence.
2 THE COURT: Any objection? Received.
3 MR. DELERY: We'd also at this point offer
4 Professor Raudenbush as an expert in the application of
5 statistical methods to education.
6 THE COURT: Any objection?
7 MR. PAYTON: No.
8 THE COURT: Okay.
9 Q. All right, Professor Raudenbush. I believe you
10 mentioned that you reviewed Dr. Larntz' work in this
11 matter.
12 A. That's correct.
13 Q. Do you have an opinion concerning, now just as a
14 summary matter, we'll get into it in more detail. But do
15 you have an opinion concerning the reasonableness of the
16 approach that Dr. Larntz took and his results?
17 A. I do.
18 Q. And what is that opinion?
19 A. I'm actually quite skeptical for two reasons. Dr.
20 Larntz attempted to construct a statistical model that
21 could tell us the extent to which race is taken into
22 account in admissions. And I'm convinced that it's not
23 logically possible to answer that question with such a
24 statistical model.
25 Moreover, certain methodological decisions made by
13
1/19/01 - BENCH TRIAL - VOLUME IV
1 Dr. Larntz, I believe, have led to a misleading impression
2 about the strength of association between minority status
3 and admissions at the law school.
4 Q. Okay. Now, you indicated that in addition to
5 reviewing Dr. Larntz' work, you did some things of your
6 own. What did you do in your analysis?
7 A. Well, as I implied, I think it's, it's not possible,
8 given the data at hand, to organize a statistical analysis
9 that's going to tell us the extent to which race is taken
10 into account in admissions. What we can do, however, and
11 what I think is very useful, is to do a causal analysis of
12 the impact of using race in admissions on those who apply
13 to the university or to the law school.
14 Q. Okay. And what are the basic conclusions again, as a
15 summary matter that you draw from your work in that
16 context?
17 A. What we did, and we'll go into some detail on this, is
18 we compared the current policy, which does use race as a
19 factor in admissions to an alternative policy that would
20 not use race as a factor. And we estimated how that
21 difference in policies would effect the average probability
22 of admission of various people who apply, various
23 sub-groups of people who apply at the University of
24 Michigan.
25 And essentially what we found, first, of course,
14
1/19/01 - BENCH TRIAL - VOLUME IV
1 is that a change in the policy would effect people
2 differently, depending on grades and test scores. It would
3 also effect people differently, depending on ethnic
4 minority status. A switch from the current policy to a
5 so-called race-blind policy would have a fairly substantial
6 effect, negative effect, on the probability of admission on
7 minority candidates.
8 On the other hand, such a change from, again the
9 current policy to a race-blind policy, would have a
10 comparatively modest effect on the positive effect, that
11 is, on the average probability of admission of majority
12 students.
13 Q. And from your work, do you draw any conclusions about
14 the likely effect on the diversity of the law school class
15 of moving to a race-blind admissions policy?
16 A. Yes. We can then take the admissions probabilities
17 under the current policy, as compared to an alternative
18 policy. And from that data, we're able to project the
19 number of applicants, not only who will be admitted, but
20 then using yield statistics, how many would then, in fact,
21 attend the law school. And then we can have an estimate of
22 how the class composition would look of the first-year
23 students at the law school. And so we're then able to make
24 some statements about the likely diversity with that class.
25 Q. And what do you conclude?
15
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. And what we conclude is that switch from the current
2 policy to a so-called race-blind policy would, would quite
3 dramatically reduce the fraction of students who are from
4 underrepresented minority backgrounds, and we'll define
5 that as we go, and to try to understand the practical
6 implications of that, we then took a look at how that would
7 translate into the composition of various contexts for
8 learning that occur in the law school. And, again, the,
9 how different classrooms and other context for learning
10 would look under the current policy versus an alternative
11 policy is really quite different.
12 Q. Well, with that sort of basic overview in mind, let's
13 go back and talk in more detail about how you arrived at
14 these conclusions.
15 A. Okay.
16 Q. What was, basically, the first thing that you did when
17 you approached these data?
18 A. Well, the first thing we did, and we did this for each
19 year between 1995 and 2000, was just to take a look at the
20 basic data; who applied at the law school, who was
21 admitted, who, how many people who were admitted decided to
22 come to the law school, and then what was the composition
23 of the first year class for each of those years.
24 Q. Okay. And I think we've prepared a chart of an
25 illustration of that, is that right?
16
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. Yes.
2 Q. Is that right? And I'd like to put up, if I could,
3 Your Honor, Exhibit 184, the series of exhibits, I think,
4 is in the supplemental exhibit file. And the lights on
5 would be fine, because they're just words today, no screen.
6 A. Your Honor, may I stand up and explain what's on the
7 screen?
8 THE COURT: You may absolutely stand up and
9 explain, yes, or we can move it closer to you so you can
10 sit.
11 A. Yeah.
12 THE COURT: You're a professor, you're used to
13 standing and talking.
14 A. That's right. Either that or I'll have to get new
15 bifocals.
16 THE COURT: Yeah, whatever.
17 A. That's fine, thank you.
18 THE COURT: I've got a pointer here if you'd like
19 one, too, however you got to promise to give it back.
20 A. Right.
21 THE COURT: Because the government, again, we can
22 get almost anything we want, but pointers. They're hard to
23 come by these days.
24 A. It will be hard to walk away with this.
25 THE COURT: Yeah.
17
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. All right. So this chart is of the 2000 admissions
2 data, is that right?
3 A. That's right.
4 Q. Why don't you explain what's here and what you find
5 significant about these numbers?
6 A. Well, the basic idea behind this chart is that it
7 shows quantitatively how a pool of applicants gets
8 translated into people who actually attend the law school.
9 And the thing to illustrate that, I'll just use
10 the top row of the chart in 2000. And we break this down
11 by ethnic groups. So just to take the first group here in
12 2000, there were 262 African-American applicants and that
13 constituted about 7.4 percent of the applicant pool.
14 And of those 262 people, 36.3 percent were
15 admitted. And that led to 95 offers of admission for that
16 group. Now, of those people who were offered admission,
17 only a minority, 40 percent, decided to come to the law
18 school. So if you multiple 40 percent times the 95 who
19 were admitted, then you get the number of African-Americans
20 who actually were attending the law school in 19, in 2000,
21 and that turns out to be 38.
22 So what you, basically, see is that this number on
23 the left which is 262, ultimately becomes 38, through whose
24 admitted and whether they decide to attend. That's the
25 basic idea on the chart.
18
1/19/01 - BENCH TRIAL - VOLUME IV
1 Now, what we've done is, is to, to make this
2 clear, and I think in conformity with the law school policy
3 of admissions, is we've taken three groups;
4 African-Americans, Hispanics and Native Americans and
5 combined their data in the lower panel here to the data, to
6 a group that we label those of underrepresented minority
7 status. So that --
8 Q. That's UMS?
9 A. And that's called UMS in this table. And then we have
10 taken data from the Caucasian group, Caucasian American,
11 and those, those whose ethnicity is unknown, and again,
12 that's in accord with our understanding of the how the
13 policy works. And we've taken their data and combined them
14 into another group that we call them non-UMS. They are the
15 ones who are not in the underrepresented minority status.
16 Now, that leads one group that I haven't
17 mentioned, and that's the other group, the non-citizen
18 group, and that group, we do not include in this table. We
19 could have looked at underrepresented minority status, yes
20 or no, and foreign or foreign students. But the numbers,
21 in fact, there were only three foreign students attending
22 in 2000, are really too small to do much with. And it
23 seemed that whatever was happening with minority status and
24 non-represented minority status was somewhat different
25 because this group is ethically very diverse, the foreign
19
1/19/01 - BENCH TRIAL - VOLUME IV
1 group, and yet they're not in these categories so we didn't
2 include that small number of applicants.
3 So then down at the bottom what we, basically,
4 have are underrepresented minority students and
5 non-underrepresented minority students and then a total.
6 Q. Do you find anything significant about the pattern of
7 the numbers here on the bottom half of the chart?
8 A. Yeah. There's several significant features of this
9 table. One is we just start by just looking at the
10 applicant pool. So we see that there are 484 applicants
11 who are minority. I'm just going to use the word
12 "minority" and "non", I think, because it gets hard in
13 saying.
14 THE COURT: That would be great. We all
15 understand.
16 A. And I'll try, and I often may use the word "race", and
17 I don't necessarily mean "race". We know there's ethnicity
18 and it's complex, but I'll use it because it gets hard to
19 use so many words.
20 But so 484 minority applicants, and in contrast to
21 2,871, majority applicants, or non-majority applicants.
22 And so the pool sizes are very different. There's a much
23 smaller number of minority applicants than non. So that's
24 one factor that we -- it's very important in
25 understanding -- the dynamics of this whole system is just
20
1/19/01 - BENCH TRIAL - VOLUME IV
1 the different sizes of that applicant pool.
2 The next feature that's very important to look at
3 is just the percentage admitted, because that's a crucial
4 factor in, who ends up being in law school. And what we
5 see is that 35.1 percent of the minority applicants and 40
6 percent of the non-minority applicants are admitted.
7 And these numbers are quite reflective of what
8 happens year to year. Only a minority of people, of the
9 overall applicant pool is admitted. The numbers are pretty
10 similar. In general, the fraction admitted is smaller for
11 the minority group than for the non-minority group.
12 Q. And is that true in each of the years from 1995 to
13 2000?
14 A. That is true. The general pattern is true each year.
15 These numbers will fluctuate but the general pattern is
16 true. And the, from the point of view of promoting
17 diversity, ethnic diversity, which is one of the goals
18 stated in the admissions policy, these two facts; there is
19 the small pool size and the comparatively small fraction of
20 those admitted has important implications for the diversity
21 of the class. Because if this number is lower much, the
22 number of people who actually attend can get very small.
23 Specifically, in this case, with 35.1 percent of
24 the minority applicants admitted, and then with the yield
25 of 34.1 percent out of the 484 applicants who are minority,
21
1/19/01 - BENCH TRIAL - VOLUME IV
1 what we see as actually attending, 58. So 484 goes down to
2 58.
3 And if you're, you know, if you're interested in
4 diversity, the size of this applicant pool, the fraction
5 admitted and the yield are going to strongly effect this
6 number, and I guess this percentage admitted is under --
7 obviously under the direct control of the law school. And
8 if we shadow where we're going with our analysis, if this
9 number were reduced significantly, this number 58 would
10 begin to go down.
11 I mean, if this number were cut in half, then we'd
12 have only 29 minority students, so, and that would assume
13 that the number of applicants and the yield would remain
14 constant. Do you see my point? That if we cut this number
15 in half, hold everything else constant, we're down to 29.
16 THE COURT: Or double it and it may go up?
17 A. Or double it and it will be go up to 116 if we double
18 it. So whatever we do here has big effects on this number,
19 but we also need to take into account the possibility that
20 changing this number could change this number. It could
21 change the number of people who apply. It could also
22 change this number, the number of people once admitted who
23 might then decide to attend.
24 So in particular, if this number were lower, this
25 number could, would likely -- it probably wouldn't stay the
22
1/19/01 - BENCH TRIAL - VOLUME IV
1 same. A more likely outcome, if you lowered the
2 probability of admission of a group, it might encourage
3 fewer people to apply. That's, we don't know. And our
4 analysis won't assume that, but the law school would have
5 to take that into account as a possibility. And lowering
6 this number might also end up lowering the yield because if
7 you, if you reduce this number substantially you would be
8 left with, under a race-blind policy, the people who would
9 be here would be extreme.
10 Q. "Here" being the number admitted?
11 A. The number admitted would be an extremely highly
12 qualified group, in terms of grades, test scores and so
13 forth. And the yield for such a group may be, may be lower
14 because there may be significant competition, among law
15 schools for those people. So changing this number could
16 impact these numbers. And with, with large effects on
17 this, this relatively small number, 58, so that's, that's
18 the key thing that's happening.
19 Q. Right. So this chart is of the 2000 data. Does your
20 report include similar information for the other years?
21 A. It does.
22 Q. For the various reports?
23 A. We have a similar flow chart for each year from 1995
24 to 2000.
25 Q. And is the 2000 data unusual, compared to the other
23
1/19/01 - BENCH TRIAL - VOLUME IV
1 years?
2 A. The 2000 data are pretty similar in virtually all
3 regards. There's one slight difference here. The yield
4 for African-Americans candidates in 2000 was 40 percent,
5 which is, which is higher than it had generally been in the
6 other years. So that number is a little higher than
7 average, but other than that it looks.
8 Q. If we compared the, this data, including the number of
9 applicants to some similar charts in Dr. Larntz' reports, I
10 think there may be some slight differences, is that right?
11 A. I looked at those numbers. The, the exact numbers are
12 not identical and I don't really know why.
13 Q. You worked from the same database?
14 A. We worked from the same database.
15 THE COURT: Are they significantly different?
16 A. They're not significantly different.
17 Q. Okay.
18 A. The patterns that I'm describing are very similar, I
19 mean, they're virtually identical in the two sets of
20 figures.
21 Q. Now, in addition to your point about how, how the
22 various percentages, in particular, can effect the number
23 attending in each year, do you take any other basic
24 conclusions away from looking at this basic descriptive
25 data?
24
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. There are a couple of other conclusions. While,
2 remember, I mentioned that a change in this percentage
3 would lead to, perhaps, fairly large changes in this
4 number; that is, the number admitted and also this number,
5 the number attending, and that's for minority applicants.
6 If changes in this number that are small would
7 have comparatively modest effects, if, let's say, half of
8 these people were rejected instead of, let's say, that
9 would be, that would be, we have 170. That would be 85
10 people. If those 85 places became available to the
11 majority students and then these 2,871 would compete for
12 those 85 places, and so that change, which is big here,
13 that is in the minority row, would have a comparatively
14 modest chain effect on the majority role, so that's one
15 additional piece of evidence from this.
16 Q. And the comparison or the percentage admitted of the
17 two groups, I think, what also might be called the average
18 probability of admission, is that right?
19 A. Yes.
20 Q. Does that comparison tell you anything about the
21 impact of considering race in admissions?
22 A. Well, this, we call it a, yeah, we call this bivariate
23 association. There are two variables. There's the race of
24 the candidate, and then there's the admission decision, and
25 when we look at these two proportions, that gives us
25
1/19/01 - BENCH TRIAL - VOLUME IV
1 evidence about that bivariate association. Is there an
2 association between race and admissions? And we see a very
3 small bivariate association, actually which favors the
4 majority applicants.
5 Now, we use, in statistics, we tend to look at
6 these bivariate associations as a first take on what's
7 going on, just simple data, there's no model, just look at
8 the data. And so we see this relationship. And in
9 conjunction with other bivariate relationships, my
10 conclusion from this was that it, it leads one to be
11 skeptical of a claim that race is a powerful predictor of
12 the admissions decision.
13 Q. Not the end of the analysis but a starting point?
14 A. It's not the end of the analysis, but, let me expand a
15 little bit. If we look at, let's say, just the association
16 between grades and admissions, there's a very strong
17 relationship, even with higher grades are more likely to be
18 admitted. If we look, and we don't have to control for
19 race to see that. We just see that relationship. If we
20 look at the relationship between test scores, LSAT and the
21 probability of being admitted, we see a very strong
22 relationship, we don't have to control for anything else to
23 see that. We look at the relationship between race and the
24 probability of being admitted, we see very little
25 relationship.
26
1/19/01 - BENCH TRIAL - VOLUME IV
1 So that, that tells us that race is unlikely to be
2 a powerful predictor of the outcome. It doesn't mean that
3 race and admissions are not related controlling for other
4 factors, but it does suggest that race will not be a
5 powerful predictor for the admissions decision.
6 Q. Okay. I think at this point you can probably take
7 your seat again.
8 A. Thank you. Your Honor.
9 THE COURT: No, just hold on to it.
10 A. Yeah, I need it again.
11 MR. DELERY: I think we may need it again.
12 THE COURT: Maybe you can move the chart just so
13 the folks in the audience can see.
14 MR. DELERY: Sure.
15 THE COURT: Great. Thank you.
16 MR. DELERY: I apologize.
17 Q. Now, in addition to the examination of the basic
18 descriptive data, what did you do as part of your analysis
19 in the case?
20 A. Well, as I mentioned, I'm convinced, and I think will
21 explain why a little later. But I'm convinced that we
22 can't develop a statistical model that's going to tell us
23 the extent to which race is taken into account in
24 admissions. What we can do and what I think is useful to
25 do is to do a causal analysis. What's the impact of the
27
1/19/01 - BENCH TRIAL - VOLUME IV
1 policy that the university has of using race in admissions
2 on the people who apply. And that causal analysis is
3 something that we can do with a minimum of assumptions.
4 And so that's what I decided to do, and I thought that that
5 would be informative.
6 Q. Okay. Have you prepared a chart to sort of explain
7 that causal connection?
8 A. Yes, I have.
9 Q. Okay. I think for this one, you can probably just
10 stay where you are with the easel where it is.
11 A. Especially with this.
12 Q. This is Exhibit 185, right, exactly with the long
13 stick?
14 A. Right, with the long stick. I don't have to get up.
15 Q. So this chart is called conception for causal link
16 between race and admissions?
17 A. Right.
18 Q. What do you mean by that?
19 A. Well, in causal analysis and statistics, the way we
20 think is that we've got, let's say, two alternative
21 treatments. We've got treatment A and treatment B.
22 Now, for each person that we're interested in, we
23 imagine the following, that that person has an outcome
24 under treatment A and an outcome under treatment B, and the
25 difference between the two outcomes is defined,
28
1/19/01 - BENCH TRIAL - VOLUME IV
1 statistically, as the causal effect of the treatment.
2 So if I, if one person has, let's say, I could
3 randomly assign a person to have surgery for heart problem
4 or I could randomly assign to have medicine, and the, and
5 the person would have one outcome under the first
6 treatment, another outcome under the second treatment.
7 Causal effect is the difference between the two outcomes.
8 So we applied that basic idea to the, to the scenario here.
9 What we have on the left, what we have up here is, is a
10 person, an applicant which and this person.
11 Q. You can tell we're not artistic.
12 A. Right. I wouldn't want to be that person, but we have
13 that person. And this person is going to apply to the law
14 school and that person might apply under policy A. Policy
15 A is the current policy, according to the admissions
16 policy.
17 And in that policy, it states a number of factors
18 that are going to be taken into account, and I guess, I'll
19 read them. I don't know if you can see them all;
20 undergraduate grades, the law school aptitude test,
21 Michigan residency, minority status, gender is, could be
22 considered, I assume as a force, a form of diversity. The
23 quality of the undergraduate school, the curriculum; that
24 is the courses that the applicant took, trend in grades,
25 not just are they how or were they going up, relationship
29
1/19/01 - BENCH TRIAL - VOLUME IV
1 with family members who are alumni. There are essays that
2 are required, letters of recommendation and leadership
3 experience. A person may have displayed other unique
4 experiences and talents and then unusual circumstances. So
5 this -- there's this list of factors that could be taken
6 into account.
7 Q. And these are all things, if I could interrupt you for
8 a second?
9 A. Yes.
10 Q. That are reflected in the policy, as you read it?
11 A. That's right. I, I got these right out of the policy
12 document itself. And so our applicant comes and applies
13 under policy A. All of these characteristics are taken
14 into account and the results is this person has a certain
15 probability of admission. We call it a probability because
16 there's some uncertainty in what's actually going to happen
17 here. There's subjective judgments being made and there's
18 some probability of admissions. So we call that
19 probability A. So that's policy A.
20 Now, if our same applicant were to apply under a
21 different policy, and we're going to call that policy B,
22 the result might different. Policy B is, we label a
23 race-blind admissions policy. And the way we're, the way
24 we're defining that is that all of the same factors that
25 were taken into account under policy A would be also taken
30
1/19/01 - BENCH TRIAL - VOLUME IV
1 into account under policy B with one exception, and that is
2 underrepresented minority status. That would not be
3 considered. So we call that a race-blind policy.
4 So our applicant comes along now, low and behold,
5 policy B is in effect. These are taken into account, these
6 factors, and the result is that our applicant has a
7 probability of admission, a piece of B.
8 And so with that scenario in mind, we can define
9 the causal effect of policy A versus policy B as being the
10 difference in the two probabilities of admission. So if,
11 let's say our applicant applied under policy A and got a
12 piece of A, probability under B, a piece of B.
13 Suppose those two probabilities were the same,
14 identical, there would be no causal effect of a change in
15 policy on that person. Suppose, on the other hand, that
16 these probabilities were very different. A person was,
17 let's say, you know, very unlikely to get in under policy A
18 and very likely to get in under policy B, big causal effect
19 of the policy. So that's, basically, how we defined the
20 causal effect. And that was what set up our analysis.
21 Q. Now, why do you think it's important to look at this
22 contrast between two policies in this case?
23 A. There are two reasons. One is that a change from
24 policy A to policy B could effect the diversity of the
25 incoming class and that's one of the goals stated in the
31
1/19/01 - BENCH TRIAL - VOLUME IV
1 admissions policy is to have an ethically diverse class,
2 and so we can use this framework to assess the effect,
3 causal effect on the change of policy on the diversity of
4 the class.
5 The other reason that it's important is that it,
6 it's a way of gauging the causal effect of, on those who
7 apply, I mean, I think that a person who applied to the
8 university, or to the law school, would be very concerned
9 about, are my probabilities going to be very different
10 under these two, under these two policies. If they were,
11 that would have important effect on behavior of people who
12 apply and it's just an important issue and it gauges the
13 extent to which the current policy is strongly effecting
14 the outcomes of people who apply.
15 Q. Okay. And is this kind of comparison between
16 alternative policies the standard way in your field to get
17 at causal questions?
18 A. This has become the, essentially, the consensus in how
19 we think about causation in statistics, two alternative
20 policies, an outcome under each for each person and the
21 causal effect being defined, as I mentioned.
22 Q. Now, how, if at all, does this conception, this
23 approach, differ from what Dr. Larntz did?
24 A. Okay. In Dr. Larntz' analysis, he's analyzing the
25 data that were generated under policy A and computing
32
1/19/01 - BENCH TRIAL - VOLUME IV
1 correlations or associations and trying to use those to
2 make strong causal inferences. And, as I mentioned, I'm
3 convinced that that's not logically possible to do in this
4 case. This kind of analysis --
5 THE COURT: You say in this case, in any case?
6 A. With, well, I think part of the problem is the amount
7 of available information. With, if, with a great deal of
8 information, one might be able to make a better, I think
9 that's an important constraining piece, if there were
10 enough information, but we really had very limited
11 information about the people who apply, numerical
12 information, so I think that's a key constraint on the, on
13 a correlational approach. Generally.
14 THE COURT: Well, you say.
15 A. Sure.
16 THE COURT: Limited numerical information. What
17 other, on your list, there's only certain things that can
18 be equated to numbers.
19 A. Right. And that's one of the difficulties in drawing
20 a causal inference from numerical data is the --
21 THE COURT: Oh, I see.
22 A. If the important, if many of the important factors are
23 not co-indentifiable.
24 THE COURT: I see. Thank.
25 A. That would be a good reason why we didn't have that
33
1/19/01 - BENCH TRIAL - VOLUME IV
1 information.
2 Q. Now, with this conception for the causal analysis in
3 mind, what did you do next in your analysis?
4 A. What we tried to do then was to compare policy A and
5 B, and I think we have an exhibit that displays how we
6 approach that.
7 Q. Okay. Let's put up Exhibit 186 now, the next chart.
8 Does this chart illustrate how you approached your
9 analysis?
10 A. It does. Simulating would happen under policy A was
11 very easy because we actually didn't have to simulate it.
12 We have the data from the years '95 to 2000. So we just
13 actually used, we used the actual reported admissions
14 results to compute probabilities of admission, average
15 probabilities, of admission for various sub-groups who
16 applied, and those were just based strictly on the data.
17 Policy B posed us with a more challenging problem.
18 We don't know what the effect will be on the probability of
19 admission under policy B, because it's never been
20 implemented. So we have to make some assumptions.
21 Essentially what we did was we had data on grades,
22 on test scores, Michigan residency and gender. And we can
23 develop, based on past data a prediction equation that
24 would predict the probability of admission, based on past
25 data. And then from that we can simulate what's happening
34
1/19/01 - BENCH TRIAL - VOLUME IV
1 under policy B. The problem we face is the same problem
2 that Professor Larntz faced. There's a lot of information
3 that we don't have. We don't know anything about the
4 undergrad school curriculum, etc., essays, recommendations,
5 all these other things, these long list of factors. We
6 don't have any numerical data.
7 Q. When you say "we don't know about those things", you
8 mean that, as a statistician looking at the data you don't
9 know?
10 A. Exactly. As a statistician analyzing the numerical
11 database, I only have access to a small fraction of the
12 relevant information used in make admissions decisions, so.
13 Q. The admissions officers have more information than you
14 have?
15 A. Exactly. And that's why, that's one of the reasons
16 why it's difficult to model those decisions. They know a
17 lot more than we do. And we have to make assumptions about
18 what we don't know. In order to do this simulation, we
19 have to assume, essentially, that all of these factors that
20 we don't know anything about are not associated with the
21 factors that are in our model.
22 THE COURT: So you have quite a few there?
23 A. That's right.
24 THE COURT: And Dr. Larntz testified that the
25 fewer assumptions you make, and I'm not saying you have to
35
1/19/01 - BENCH TRIAL - VOLUME IV
1 agree or not agree, but I'd like your opinion on it. He
2 testified that the fewer assumptions you make, the better
3 your results are. That when you start making assumptions,
4 that it may skew it to subject -- I don't think you used
5 the word, subjective, but at least it's more extensive. In
6 your model you're making assumptions, at least, as to one,
7 two, three, four, five, six, seven, eight, nine, ten areas?
8 A. That's right, exactly.
9 THE COURT: So do you disagree with him?
10 A. Oh, I agree with him on that, absolutely, yes. We're
11 very concerned about the impact of the possible falsehood
12 of these assumptions. And there are almost certain to be
13 some falsehoods here. The question is the falseness of
14 these assumptions, the question is to what extent does that
15 effect the result.
16 We know we're not going to really have the model
17 right, but to the extent we have it wrong, to what extent
18 does that have some effect on our results. And that's what
19 we then had to do in this was to, what we actually did was
20 we did this simulation.
21 We looked at the results. We repeated the
22 simulation a couple of other ways, but actually, this is,
23 in some ways that I believe the great strength of the
24 causal analysis. We can put bounds on the errors of your
25 our estimates that require virtually no assumptions, so we
36
1/19/01 - BENCH TRIAL - VOLUME IV
1 can actually assess the extent to which errors in our
2 assumptions effect our results in a very sure-minded way,
3 and I'll try to explain how we did that as we go.
4 So the way, the way it works is, is you do an
5 analysis, based on assumptions, you look at the results,
6 you try another analysis, generally, that's based on maybe
7 some different, slightly different assumptions. But then
8 you try to bound the error in your results as a function of
9 your assumptions, and we we'll show how we do that.
10 MR. DELERY: I think it will be easier to see
11 that, Your Honor.
12 THE COURT: That's fine.
13 MR. DELERY: After we see the results.
14 Q. But before we leave this point, while we're on
15 assumptions and just so we're clear, what, what is, or what
16 are the assumptions about the factors below the line on the
17 chart, as related to the factors above the line, just so we
18 have that in mind?
19 A. Right. Basically the assumption is that if any of the
20 factors below the line are correlated with, with the
21 factors above the line, then our estimate of the effects of
22 the factors above the line will be biased.
23 Q. So --
24 A. And if they're biased, the predictions, the predicted
25 probabilities will be potentially biased as well.
37
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. I think we'll come back to that as to how you dealt
2 with that, is that right?
3 A. Yes.
4 Q. All right. But before we go to look at the results,
5 let me just ask you a couple questions about exactly what
6 you did. Did you, just as a general matter, did you use
7 any particular kind of, of analysis to undertake the
8 simulation?
9 A. We did. We used -- the first method we used was
10 called, logistic regression. And I think we've had a
11 discussion of that. You have a binary outcome which is
12 admitted, yes or no, and then you have a number of what we
13 call explanatory variables, which are the ones here above
14 the line. And you are able to estimate an equation that,
15 that estimates the relative weights of these factors on the
16 probability, the log odds of admission, and ultimately we
17 can translate like that into the probability of admission.
18 Q. So -
19 A. We've talked about that in court. And I assume we
20 don't need to necessarily say much more about it. I think
21 Professor Larntz explained what that was.
22 Q. And so Dr. Larntz also used logistic regression, of
23 course, as part of his analysis?
24 A. Yes.
25 Q. And we'll get back to Dr. Larntz' regression models.
38
1/19/01 - BENCH TRIAL - VOLUME IV
1 But are there general things that you can say about how
2 your regression analysis differed from, in addition to the
3 conception from what Dr. Larntz did?
4 A. We, yes. We actually estimated our models separately
5 for minority and majority applicants. And the reason we
6 did that was that we found that the association between
7 minority status and admissions was strongly dependent on
8 grades and test scores; that is, we found that, for
9 example, applicants who had very high grades and test
10 scores, for those applicants minority status has a very
11 small effect, or very small association. And for
12 applicants in other cells the association is considerably
13 stronger. So because the association between minority
14 status and these factors varied, what statisticians then do
15 is, they say we can't estimate one model for everybody, we
16 then do the models separately.
17 Q. Did you exclude any of the applicants for which you
18 had data from your analysis?
19 A. No. We used -- oh, I should say, we did exclude
20 people, a very small number of people have have no grades.
21 There's just, they don't have grades in the database. It's
22 a tiny fraction, or they don't have LSATs, so those people
23 we excluded. But we excluded no cases based on their
24 outcomes.
25 And this is a very important point. When you
39
1/19/01 - BENCH TRIAL - VOLUME IV
1 start excluding cases from an analysis based on the outcome
2 of the admissions decision, you get into some significant
3 biases and we did not do that.
4 (Whereupon an off-the-record
5 discussion was had.)
6 Q. All right. So with the simulation model or regression
7 model, how did you conduct your simulation?
8 A. So what we did was we actually, for each year, we did
9 the analysis I mentioned, we did it separately for majority
10 and minority applicants. We actually used the majority
11 equation in predicting the probabilities of admission under
12 the race-blind policy. We assumed that under the so-called
13 race-blind policy that the majority equation, which has
14 more cases involved in the estimation would be more like, I
15 mean, the average equation would be more like that. So we
16 used that equation.
17 Q. Okay. And with that equation, what did you do?
18 A. Well, based on that equasion we could compute the
19 predicted probability of admission under policy B for any
20 applicant, and then we could combine those within ethnic
21 groups to predict the average probability of admission for
22 any sub-group of applicants in this case, as a function of
23 ethnicity.
24 Q. And so from that you can estimate how, what the
25 percentages admitted would look like?
40
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. Exactly. From that we're able to compute the average
2 probability of admissions for ethnic, for minority and
3 majority applicants, and compare it to the observed
4 probability of admission under the current policy.
5 Q. All right. I'm going to ask just one other thing
6 about the simulations. Are you able to, are you able to
7 say, based on the simulation, what would happen to any
8 particular applicant under the alternative policy?
9 A. No, we're not. And this is one of the ironies of
10 causal inference and causal modeling. For any person,
11 we'll never know the two probabilities. In order to do
12 that -- we can't even imagine how to do it. We'd have to
13 have both policies in operation and we'd have to have them
14 implied under both policies and see all the results. But
15 we can't do that. And that's generally true in causal
16 inference. We can't compute the causal effect for any
17 specific case. What we can compute is called the average
18 causal effect. In this case, it would be the average
19 probability of admission under policy A, minus the average
20 under policy B for sub-groups of applicants.
21 Q. Now, let's look at, if we could what happened in your
22 simulations. I think the next Exhibit is 187 in the same
23 category.
24 A. Now, mind you --
25 Q. Yeah, why don't you first tell us what the columns
41
1/19/01 - BENCH TRIAL - VOLUME IV
1 are.
2 A. Right.
3 Q. And then --
4 A. Yeah.
5 Q. Explain what the results are?
6 A. Let me just preface it by saying that the results of
7 policy B are going to be those based on the model I just
8 described, but we also replicated this analysis using
9 another, actually a couple of different regression models
10 we tried. But we also used another method, which we can
11 describe a little bit later. But under the method that I
12 just described --
13 Q. Can I, let me just ask you --
14 A. Yeah.
15 Q. Are the results under the other methods substantially
16 different?
17 A. They're not substantially different. They're somewhat
18 different but in the main, they're very, very similar.
19 Q. All right. So why don't you explain what you have on
20 the chart and then what the results showed.
21 A. Okay. What we have on the chart are two columns,
22 policy A, again, that's the current policy; policy B, this
23 is the so-called race-blind policy that I mentioned.
24 Q. And just so we're clear, the number in policy A is the
25 actual observed data?
42
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. Right. And so we have for minority and non-minority
2 applicants, and for each year, the predicted -- well, in
3 this case under policy A, the actual observed average
4 probability of admissions. And then under policy B, the
5 average probability of admission for that same group.
6 A. So again looking at 2000, we've been looking at 2000.
7 The average probability of admission in 2000 for minority
8 applicants was .35. We project that under policy B the
9 average probability of admissions would be .10, which is,
10 which is quite a large difference. And that type of result
11 occurs in each year. They're pretty similar. There's some
12 exceptions.
13 It turns out that 1995 is a bit extreme in terms
14 of the change in the probabilities for the minority group.
15 But, but it follows the same pattern. It's, and the other
16 years are very similar to, to the year 2000. So we see
17 then in some, a quite sharp reduction in the average
18 probability of admission of the minority applicants under
19 policy A and policy B.
20 A. Now, if we move down to the bottom panel, we have the
21 results for the non-minority applicants under each year.
22 So, again let's just take a look at, for illustration of
23 the year 2000 under policy A the average observed, average
24 probability of admission was .40, 40 percent of those who
25 applied were admitted. We project that under policy B,
43
1/19/01 - BENCH TRIAL - VOLUME IV
1 this is a race-blind policy, that would increase. It would
2 increase from .40 to .44. So it would be rather marked,
3 small or marginal increase in the average probability of
4 admission, .40 to .44.
5 Q. And are the results similar for the other years?
6 A. And the results are very similar for other years. It
7 tends to be, .99 goes 41 to 45, again the difference being
8 .04. In some cases it's .05. I think the actual biggest
9 one we see is in '95, not surprisingly, which is .06, .28
10 up to.34.
11 Q. Now, why is it, Professor Raudenbush, that the change
12 in the average probability of admission is fairly large for
13 the minority students and fairly small for the non-minority
14 students?
15 A. It's a very straight-forward result of the difference
16 in the sizes of the applicant pools. There are relatively
17 few minority applicants, a small -- a large change in the
18 probability, a large reduction in the probability of
19 admission of those candidates translates into a very small
20 increase in the probability of admission of the majority
21 group, because it has so many more applications; basically,
22 any extra, sort of admission seats, if you will, or admits,
23 could become available by reducing this probability, will
24 be competed for by a large number of people.
25 Q. Now, as Judge Friedman alluded to earlier.
44
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. Right.
2 Q. These simulation results are based on regression
3 models which involve assumptions, correct?
4 A. Right.
5 Q. How can you be confident, given those assumptions
6 about these results here?
7 A. Right. Well, the first thing we did, as I mentioned,
8 was we did use an alternative method to do the simulation,
9 and as you asked me were the results similar and the answer
10 was, yes, they were very similar.
11 Q. So the fact that you got similar results says what
12 about these?
13 A. From an approach that did not use logistic regression
14 at all, and I'll explain exactly what we did a little bit
15 later. But the most important way that we can bound our
16 error, if you will, is much more straight forward and
17 requires an absolute minimum of assumptions. And I think I
18 can maybe demonstrate that with a different exhibit.
19 Q. All right. Why don't we go.
20 THE COURT: Let me ask you one question?
21 A. Sure.
22 THE COURT: You can also conclude from that chart
23 that by having a race-blind policy that, looking at 2000,
24 for example, that there's a 25, obviously a 25 percent
25 difference, so that.
45
1/19/01 - BENCH TRIAL - VOLUME IV
1 A. Right.
2 THE COURT: That's right. So you could also say,
3 could you not, that the effect is, the effect having a
4 policy that's not race blind is about 25 percent?
5 A. A difference in probabilities of .25, yes, right. And
6 people do this in different ways. We talk about odds,
7 ratios of probability. Sometimes differences in
8 probabilities are the most straight-forward way of
9 interpreting the results. It depends on the situation.
10 Q. Let me ask a related question.
11 THE COURT: Well, go on.
12 MR. DELERY: Please.
13 THE COURT: You ask, I'll get mine later. He may
14 answer. If he doesn't.
15 Q. Before we look at the bounding point, in your view, do
16 these numbers here, the results of your simulation analyses
17 say anything about the extent to which race is considered
18 by admissions officers in making their decisions?
19 A. They don't.
20 Q. And why is that?
21 A. Let me explain what could generate this difference in
22 probabilities. If you have a large, a much larger
23 applicant pool than can be admitted, so you have many more
24 people apply than you can accept; and if grades and test
25 scores are very important, play an extremely important role
46
1/19/01 - BENCH TRIAL - VOLUME IV
1 in the admissions decision, then a very small difference
2 between two groups can lead to a large difference in the
3 probability of being admitted.
4 And so under this, under our simulation of the
5 race-blind policy, grades and test scores are playing a
6 very important, and extremely important role because we
7 don't have any other data, basically. We know that there
8 are many more applicants than there are seats. And we know
9 that there's a small difference between minority and
10 non-minority applicants. And that explains why this
11 difference turns out to be big.
12 Q. So.
13 A. And it doesn't.
14 THE COURT: Turns out to be big?
15 A. Big, yes, these numbers are quite different. That
16 doesn't depend on how heavily the admissions officer weigh
17 race. It's a function of the fact that you're heavily
18 weighing a factor on which two groups have a different
19 mean.
20 THE COURT: A different what?
21 A. A different mean, a different average.
22 THE COURT: Mean.
23 A. Right.
24 Q. Just so I have a sense of the terminology here, is it
25 your view that there's a difference between measuring the
47
1/19/01 - BENCH TRIAL - VOLUME IV
1 effect or impact of the policy on the one hand?
2 A. Right.
3 Q. And the extent to which a particular factor is
4 considered in an admissions process on the other?
5 A. There's a great deal of difference. And I might add,
6 especially in this case, the causal impact of the policy is
7 much more excessible to statistical investigation than is
8 an attempt to discern how people who are making decisions
9 about admissions are weighing one of many factors, when we
10 don't have any information about most of the factors. It's
11 just a very difficult thing to do, statistically. We
12 basically can't do it.
13 So, but we can assess the impact of what they do.
14 We don't know why it has that impact. You see, there's a
15 big difference between finding a causal effect and
16 explaining the causal effect, knowing why it happens.
17 There are lots of things in social science,
18 medical science, where we know there's an impact on
19 something, but there's so many possible explanations. And
20 we don't have the information to explain the explanation.
21 So this analysis can be conducted with a minimum of
22 assumptions and with a considerable amount of confidence,
23 whereas the more, the much more challenging task of trying
24 to use statistical information to discern how people who
25 have much more information than we do, how they think.
48
1/19/01 - BENCH TRIAL - VOLUME IV
1 This is much more difficult.
2 Q. I think we'll come back to this question of extent a
3 little bit with some additional illustrations, but let's
4 return to the bounding point?
5 A. Right.
6 Q. That you were on, if we could. And I think the next
7 exhibit is 188.
8 MS. MASSIE: Judge Friedman, I don't know if this
9 is, if we could take a quick break, that would be great.
10 THE COURT: Of course, how much do you want?
11 MS. MASSIE: Five minutes.
12 THE COURT: Okay. We'll take a five-minute break.
13 (Whereupon an off-the-record
14 discussion was had.)
15 THE COURT: Okay. You may be seated. Thank you.
16 MR. DELERY: Thank you, Your Honor.
17 Q. Professor Raudenbush, I believe we had been talking
18 about the simulation results for the minority students on
19 the one hand and the non-minority students on the other
20 hand and the bounding issue that you?
21 A. Yes. Just to recreate where we were, the key result
22 here was that the effect of going from policy A to policy B
23 was quite big for the minority students. Like in 19, in
24 2000 it was 25 percentage points, whereas the effect going
25 from policy A to policy B on the non-minority students was
49
1/19/01 - BENCH TRIAL - VOLUME IV
1 quite small.
2 So in 2000, going from forty, .40 to .44, so going
3 up on four percentage points. So that's where we were, and
4 the question is the problems with this model.
5 As we discussed, policy A, policy B is based on a
6 simulation. It's based on a model. The model has to make
7 assumptions. The assumptions, not might be, but probably
8 are wrong, and so how far off might we be, as a result of
9 failure of those assumptions, and that was our next step.
10 Q. Okay. And here, are you talking about the assumptions
11 that the factors not in the model are unrelated to the
12 factors in the model?
13 A. Correct.
14 Q. Did Dr. Larntz' model include the same assumptions?
15 A. Yes.
16 Q. Well, why don't you move to the next chart, actually,
17 and tell us what you did. The next chart will be 188.
18 Tell us what you did to evaluate how reasonable your
19 results were, in light of the assumptions.
20 A. What we did was we used an idea that has a fancy name
21 but it's a real simple idea. The fancy name is, these are
22 non-parametric upper and lower bounds on causal effects.
23 The simple idea is how, how small could the effect be and
24 how big could it logically be. And here's how simple it
25 really is.
50
1/19/01 - BENCH TRIAL - VOLUME IV
1 Again, let's just focus on 2000. And we're
2 looking at majority students here. And we see that in 2000
3 40 percent of them were admitted. How small could the
4 effect be of going to policy B? Well, logically it seems
5 that the smallest the effect could be would be there
6 probability would stay the same.
7 In other words, we go to a race-blind policy and
8 there's no impact. It goes from .40 to .40. It logically,
9 it logically can't really go down. It's hard to imagine
10 how eliminating race as a factor would make things worse
11 for, for majority students. So .40 is the lower bound for
12 the effect. So zero percentage points, .40 to .40. The
13 upper bound is, is constructed, again, very simply; how big
14 could the effect be. The biggest it possibly could be
15 would be if every minority students were rejected under
16 policy B. If you eliminate race as a factor and every
17 single minority students were rejected, then that means
18 that's the biggest effect it could be.
19 And under that scenario, the upper bound is .46,
20 so that means the difference between the lower bound and
21 the upper bound is .06. That's six percentage points. Our
22 estimate, based on our simulation is .04. It's kind of in
23 between the lower bound and the upper bound. So our .44 is
24 undoubtedly wrong, to some degree, but to what degree can
25 it be wrong, the upper and lower bound tell us, it can't
51
1/19/01 - BENCH TRIAL - VOLUME IV
1 be -- the lower bound is a .04 error, the upper bound is a
2 .02 error and those bounds don't require me to make any
3 assumptions about what's in the model, what's not in the
4 model. Those are logical upper and lower bounds.
5 Q. So based on the bounds that you found and, as compared
6 to the simulation results, do the bounds give you
7 confidence in, in your models and in your analysis?
8 A. They give us confidence in the causal effect of the
9 policy change on the majority students.
10 Q. And that's what this chart shows?
11 A. That's what this chart shows. Now, I should add that
12 the bounds on the causal effect for the minority students
13 are wider because like when, I think in 19 -- in 2000 we
14 went from, I think it was something like .34 to ten. The
15 extreme bound would be to zero. So from .34 to zero. So
16 they were a little bit wider.
17 There's a little more uncertainty as to how the
18 switch in policy would effect the minority students. But
19 there's a great deal more -- I should say a great deal less
20 uncertainty about how the change in policy would effect the
21 majority students.
22 Q. Did Dr. Larntz do any kind of similar bounding
23 analysis on the results of his regression model?
24 A. I didn't see any evidence of it in the reports. And I
25 didn't hear him put an upper and lower bounds or a
52
1/19/01 - BENCH TRIAL - VOLUME IV
1 confidence interval on the odds ratios.
2 A. By the way, a confidence interval is a weaker bound,
3 much weaker than a non-parametric up upper and lower bound
4 because this bound has virtually no assumptions. The only
5 real assumption I'm making is that going from policy A to
6 policy B wouldn't hurt the majority students, and that seem
7 indisputable.
8 Q. So these results tell us what the expected
9 probabilities of admission are for, on this chart, the
10 majority students and on the earlier chart also, the
11 minority students?
12 A. Correct.
13 Q. Did you take that analysis any further?
14 A. Yes, I did. Once we have predicted probabilities of
15 admission or average probabilities of admission for
16 sub-groups, we can then develop a picture of what the
17 composition of the first-year class would look like under
18 policy B. Of course we already know the composition of the
19 class under policy A. It's what we observed.
20 And to do this is really very straight forward.
21 We take the probabilities of admission under policy B. We
22 multiple that by the yield which is what fraction of people
23 who were admitted decided to come to Michigan, the one that
24 was actually observed. And that can then give us the
25 expected number of people in each, of each group for each
53
1/19/01 - BENCH TRIAL - VOLUME IV
1 year.
2 Q. Okay. I think we have a chart showing those results.
3 A. Yes.
4 Q. It's Exhibit 129. Just so I'm clear about your last
5 point, Professor Raudenbush, you're assuming in this part
6 of the analysis that the yield rate would not change?
7 A. That's correct.
8 Q. If the university moved to a race-blind admissions
9 policy?
10 A. Exactly. We're, it could arguably go down if this
11 change were made, in which case our results would
12 understate the impact on diversity.
13 We're also assuming, as years go by, that the size
14 of the minority applicant pool would not be effected by a
15 sharp reduction in the probability of admission, which is,
16 which is another conservative assumption. It seems
17 reasonable that if the probability of admission goes down,
18 the number of people who would take the time and effort and
19 pay the price of climb might well go down, but we didn't
20 assume that that would happen.
21 Q. Why don't you look at this chart, Exhibit 189, and
22 tell us what it shows about this next step of your
23 simulation analysis?
24 A. Okay. Again, it's divided. As we go down the, down
25 the rows, we see the years. We have under policy A and
54
1/19/01 - BENCH TRIAL - VOLUME IV
1 under policy B and in each case what's in here is the is
2 the composition of the class. So for policy A it's going
3 to be the actual composition that happened in that year.
4 Under policy B, it's what we would predict, based on the
5 simulation.
6 And again, why don't we just, for illustration,
7 stick with 2000. Under the current policy, 170 minority
8 students were admitted and based on the yield, 58 actually
9 attended. And that was, that turned out to be 14.5 percent
10 of the class.
11 Q. Those numbers were taken from the first chart that we
12 saw today?
13 A. That's right. Those are just the actual observed
14 numbers. Under policy B, we, we would predict that only 46
15 minority students would be admitted. And then applying the
16 yield, that would lead to 16 attending. So only 16
17 minority students, from 58 down to 16, and then that would
18 be four percent of the class, so our, our analysis would,
19 would predict a reduction in the fraction of students who
20 are minority from 14.5 percent to 4.0 owe percent.
21 Q. So what, if anything, do these results, I guess I
22 should back up and ask, is 2000 unusual in this respect,
23 or?
24 A. The basic pattern of 2000 appears each year. We see
25 very similar results. Again, there's a little more extreme
55
1/19/01 - BENCH TRIAL - VOLUME IV
1 result in 1995, but it's basically in the same direction,
2 same pattern, and the other years are very similar.
3 Q. So what did these results tell you, if anything, about
4 the expected diversity of the law school class under a
5 race-blind admissions system?
6 A. Right. So we did see that under this simulation, that
7 the overall composition of the class, which, in 2000 was
8 14.5 percent minority, would be very substantially less
9 diverse with only four percent of the students being from
10 minority background.
11 Q. I think you indicated that there would be somewhat
12 over a hundred fewer minority students admitted, your model
13 predicts, under the alternative race-blind policy?
14 A. Right.
15 Q. What would happen to the spaces in the class that, I
16 guess, those students had accounted for under the current
17 policy.
18 A. Right. Well, --
19 MR. KOLBO: Object to the form, basis, Your Honor.
20 THE COURT: I think it's a pretty obvious answer,
21 but why don't you rephrase it.
22 MR. DELERY: Okay. I'll rephrase it.
23 Q. Can you tell us anything about what the model predicts
24 about where the hundred-plus spaces that had been under the
25 current policy given to admitted minority students? What
56
1/19/01 - BENCH TRIAL - VOLUME IV
1 would happen to those spaces under your alternative
2 simulation?
3 A. Right. Under our alternative simulation, those places
4 which look to be approximately 134 places would be competed
5 for by all of the non-minority students; that is,
6 approximately three, 2,800, whatever the number was, of
7 students that would compete for those places. That's the
8 way we've constructed the simulation.
9 Q. Okay. Now, using these numbers, the predicted
10 composition of the law school class as a whole under your
11 alternative policy, did you do anything to look at how that
12 would translate into the more day-to-day activities of the
13 law school?
14 A. Yes, I did. And I believe we have an exhibit that
15 displays that. Essentially, what we --
16 Q. Why don't we put the exhibit up, if we could.
17 A. What we did, while that's being put up --
18 Q. -- This is 190, by the way.
19 A. People at the law school supplied me with a list of
20 some of the important contexts for learning that arise at a
21 law school. They're listed here and they range in size.
22 The first-year section is the biggest one, 85 students are
23 in the first-year section where students take many of
24 their, several of their required classes. The smallest is
25 a moot court team which is just pairs of people in a moot
57
1/19/01 - BENCH TRIAL - VOLUME IV
1 court, and, and there are other contexts. Each one has a
2 size. And what we did next was to ask questions about the
3 likely composition of these contexts for learning under
4 policy A, which is the current policy again; and policy B.
5 And that's essentially what we did. And I think we have an
6 exhibit that displays the results.
7 Q. Okay. In your view, these, these contexts were
8 representative?
9 A. I was told by the people that supplied these, actually
10 through your office, that these were the representative
11 contexts. And they cover the range of sizes of various
12 contexts. And what's really important from the point of
13 view of statistics here is the size of the context and how
14 does that then look, in terms of its ethnic and
15 composition.
16 Q. Why don't we put up the next chart, if we could.
17 That's 191. What does this chart represent, Professor
18 Raudenbush?
19 A. Okay. So what we've been done is asked questions
20 about the expected composition of each learning context,
21 from the standpoint of a majority student and from the
22 standpoint of, we just picked African-American students We
23 wanted to have a definite type of person, rather than a
24 minority student in mind when we thought about this. And
25 we didn't do it for all of the contexts.
58
1/19/01 - BENCH TRIAL - VOLUME IV
1 We picked three represent -- three that were sort
2 of across the range of sizes. We picked the first-year
3 section, which has 85, then the second row is the half
4 section. And then the residential dormitory entryway.
5 This is an entryway of a dormitory and approximately 25
6 students would be in that entryway.
7 Q. And the results for the other contexts are reflected
8 in your report?
9 A. They're in my report, right. And I think you, this
10 basically captures what's going on here. I don't think
11 it's necessary to go through all these numbers. I might
12 just pick one of them and kind of explain. The first-year
13 section, the biggest context, let's take it from the point
14 of view of the majority student.
15 What's the probability that that would be
16 segregated in the sense that that would be no minority
17 students under policy A and policy B. And the answer is
18 it's a very small like likelihood. Under either policy
19 it's unlikely that there would be no minority students.
20 It's actually .00 versus .03.
21 But then let's ask another question, well, what's
22 the probability that there would be at least, at least
23 three minority students. And it could be nearly certain,
24 which is, approximately, pushing toward 1.0 under policy A,
25 whereas under policy B that would only happen two thirds of
59
1/19/01 - BENCH TRIAL - VOLUME IV
1 the time. There would be a one-third chance of not having
2 as many as three in that section.
3 And then for, what's the probability that there
4 would be, at least three African-American students and at
5 least three Hispanic students in that group of 85. Under
6 policy A it's almost certain to occur. Under policy B,
7 approximately one time out of four. So it's actually not
8 likely to have that agree of diversity. That's the biggest
9 section. The effects of the policy are more pronounced
10 when we go to smaller-size sections.
11 For example, for example, just take, take the, the
12 residential dormitory, what's the probability of having at
13 least three minority students, .75 in that residential
14 dormitory, to picture, 25 people who live in the dormitory,
15 .75 probability that at least three of those people would
16 be minority under policy A. Under policy B, .08, a very
17 unlikely matter. So that kind of demonstrates what's going
18 on from the point of view of the majority student.
19 Things are a little bit different from the point
20 of view of an African-Americans student because, you know,
21 the African-American has to be in the context before we can
22 ask what's happening. So given that there is an
23 African-American, we ask questions, the following
24 questions; what's the probability that you'd be the only
25 African-American student in that context, or, you know
60
1/19/01 - BENCH TRIAL - VOLUME IV
1 what's the probability of three or more of those.
2 So just, we could say, again, take, take the
3 residential dormitory example, under policy A, that's the
4 current policy -- there's a pretty small chance that you'd
5 be the only African-American student, .18, in this
6 residential dormitory. Under policy B, .69, it's very
7 likely that you would be the only African-Americans student
8 in the dormitory. And the probability of at least three,
9 at least two other African-American students would be,
10 would be relatively high under policy A, .56, at least
11 better than half, and very low, .07, under policy B.
12 So I think this gives some flavor of our
13 expectations about what would happen to the diversity of
14 certain contexts for learning under a change in policies.
15 Q. All right. Now, taking all of these simulation
16 analyses together, the overall picture that you've
17 presented here this morning, what conclusions, if any, do
18 you draw about the impact of using race in law school
19 admissions at the university?
20 A. I draw several conclusions. The first is that the
21 impact on the probability of admission of minority
22 candidates would be quite substantial. There would be
23 quite a sharp reduction in the probability of admission.
24 The second conclusion would be that the impact on majority
25 applicants would be modest, by comparison. There would be
61
1/19/01 - BENCH TRIAL - VOLUME IV
1 a small increase in the average probability of admission
2 for majority candidates. And about that conclusion, I feel
3 considerable confidence.
4 Q. And again, why do you think there is that difference?
5 A. And the reason that that's, that difference occurs,
6 that is, you know, why does it effect minority students
7 more than majority students, it's simply a result of the
8 smaller pool of applicants of the underrepresented minority
9 group than of the majority group.
10 Q. Now, so by giving these views and these estimates of
11 the impact of considering race and admissions, are you
12 saying anything about the extent to which the race of an
13 applicant is considered by the admissions people?
14 A. No. We're not making any inferences about how heavily
15 this is being weighed by the people who are making the
16 admissions decisions. We don't have information about that
17 question. But we do have information about the impact.
18 Q. And are these impacts, estimates, telling anything
19 about the relative weights of any of the factors in the
20 admissions process?
21 A. No. They're not quantifying the relative weights of
22 anything in the process.
23 Q. Okay. So as I think you indicated before, this
24 simulation results, simulation analysis, I should say, is a
25 different approach from the approach that Professor Larntz
62
1/19/01 - BENCH TRIAL - VOLUME IV
1 took?
2 A. Correct.
3 Q. Is that your view?
4 A. Correct.
5 Q. Is that your view?
6 A. That's right.
7 Q. How does your simulation analysis bear on an
8 evaluation of Dr. Larntz' work?
9 A. Well, I think that the simulation analysis gives a
10 framework of a policy framework. We've looking at policy
11 options faced by the law school that we can use to
12 understand the reasonableness of some of the results
13 results of Professor Larntz' work.
14 Q. And in your opinion does Dr. Larntz' work provide an
15 accurate or realistic picture of the role that race plays
16 in law school admissions?
17 A. And of course the answer is, no. As I stated at the
18 outset, Professor Larntz attempted to construct a
19 statistical model that could tell us the extent to which
20 race played a role. And I don't believe that we have
21 information that can enable us to do that.
22 Q. And on its own terms, do you believe that Dr. Larntz'
23 approach was appropriately executed?
24 A. Well, I believe that certain key methodological
25 choices that Dr. Larntz made led to a, an exaggerated
63
1/19/01 - BENCH TRIAL - VOLUME IV
1 impression about the association between minority status
2 and admissions.
3 Q. And what were those?
4 A. Well, they're essentially --
5 Q. Just briefly and then we'll get into them a little
6 more?
7 A. I'll give you three types, and I know we'll talk about
8 some of the details.
9 The first was that his analysis selectively
10 attended to the data; that is, it discarded data based on
11 the outcomes of the admission process. And it discarded
12 data that was, in fact, discrepant with the hypothesis that
13 there is a strong correlation between race and admissions.
14 That was the first.
15 The second was that his analysis was based on
16 strong assumptions, as our policy via regression, as I
17 explained the same kinds of assumptions that we had.
18 And that in one important case, I did an analysis
19 that showed that a key assumption that he made and was an
20 important one, was not true. And in the second case, the
21 other, another key assumption is like what I described
22 before. It's probably not true, almost certainly not true,
23 the problem being we don't know the impact. We can't gauge
24 the impact of the falsehood of the assumption on the
25 validity of the results.
64
1/19/01 - BENCH TRIAL - VOLUME IV
1 And thirdly, the results of his analysis were
2 extremely unstable. They were very different from year to
3 year, and the size of the differences from year to year
4 really can't be explained by the process, or by the data at
5 hand. And so my conclusion is that there are aspects of
6 the methodological approach that create the instability,
7 not the admissions policy or the data.
8 Q. Before we talk about those problems that you found
9 with Dr. Larntz' work in more detail, I'm wondering if you
10 could give us a sense of, of how his overall approach, his
11 conceptual framework differed from your's?
12 A. Right. Well, his, his conceptual framework was,
13 again, the idea of constructing a model that would tell us
14 about the role of admissions, the extent to which they're
15 taken into account by the admissions people, which I view
16 as a very challenging thing. You have to have tremendous
17 amount of information to assess peoples' thinking and the
18 extent to which they're weighing factors. My question is
19 actually a more limited one but one that I think we can
20 approach with minimal assumptions through statistical
21 inference and still get some very useful information. It
22 doesn't tell you, it doesn't give us the answer to that
23 question, but it gives us extremely important information
24 about the impact of taking race into account.
25 Q. Now, obviously you were here the other day when Dr.
65
1/19/01 - BENCH TRIAL - VOLUME IV
1 Larntz testified, and there was a lot of discussion about
2 odds ratios, yes.
3 A. Right.
4 Q. Obviously we all remember that.
5 A. Right. I'm just glad I don't have to explain what
6 they are.
7 Q. Is, well, I'm going to ask you to give some examples
8 in a second.
9 A. Okay. I couldn't get out of that one.
10 Q. No such luck. I guess my first question, though,
11 about this is, is computing odds ratios an accepted method
12 of statistical analysis?
13 A. It is. It's widely accepted. It's widely used.
14 Q. And in what context is it appropriately used?
15 A. Well, the thing about odds ratios is that typically an
16 odds ratio by itself doesn't tell us what we need to know.
17 It's a piece of information. But to interpret the meaning
18 of the odds ratios, we, odds ratios, we really need to know
19 something about the probabilities that went into computing
20 the odds ratio because depending on what the probability,
21 you know, an odds ratio controls a function of the
22 probabilities for each group. And depending on what those
23 two probabilities are, the odds ratio could be very, very
24 different things. So my, my general rule of thumb is to
25 always keep in mind the probabilities as well as the odds
66
1/19/01 - BENCH TRIAL - VOLUME IV
1 ratios, for that reason.
2 Q. And you have used odds ratios in your work?
3 A. Oh, yes.
4 Q. Is that right?
5 A. Yes, I have.
6 Q. Okay. In your opinion, do odds ratios provide an
7 accurate or appropriate way to look at the role that ratios
8 make in the law school admissions process?
9 A. There's some problems with using, there's some huge
10 problems with using them alone, again, without, without
11 accompanying them with other information. Generally what
12 happens to the odds ratio is that it becomes very unstable
13 when one group or the other has a probability or, of either
14 nearly one or nearly zero.
15 Q. Do you have some illustrations of that effect?
16 A. Well, I thought we might actually just revisit some of
17 the odds ratios we looked at. Was that, the day before
18 yesterday I think it was, right. The day before yesterday.
19 And maybe we could even just quickly review those. I don't
20 know if we still have those charts or if we need to
21 scribble down those things again.
22 Q. I think we do. I think the page that we have before
23 is gone.
24 A. May I.
25 MR. DELERY: I'll move the easel out a little bit
67
1/19/01 - BENCH TRIAL - VOLUME IV
1 here.
2 A. Thank you. I think what we had the other day was we
3 had a group, some group. Let's call this group one, that
4 had a probability of admission of .99. And then we had
5 group two that had a probability of admission of .90, and
6 the odds ratio turned out to be eleven.
7 So, basically, this was saying group one had
8 eleven times the odds of admission of group two. And then
9 we had another example where group one had a probability of
10 admission of .999. Group two still had a probability of
11 .90. And what happened to the odds ratio was that it
12 became 111. And then just, you can see the pattern here.
13 If group one had a probability of admission of .9999 and
14 group two system had a probability of admission of .91, the
15 odds ratio went to 1,111. Now, those are, those are facts.
16 There's no problem with that.
17 The only problem is, if all we saw, if I hid these
18 probabilities, and all I saw were the odds ratios, I might
19 get the impression that those are three extremely different
20 results. Eleven times the odds, 111 times the odds, 1,111
21 times the odds. These look so different. But when I look
22 at the probabilities of admission from a practical point of
23 view, if I'm a candidate, and my probability is .99 versus
24 .90, that's about ten percentage points.
25 And I'm nearly certain to be admitted. If I go up
68
1/19/01 - BENCH TRIAL - VOLUME IV
1 to .999 versus .91 it's still about ten percentage points.
2 I'm still nearly certain, but yet my odds ratio went up by
3 ten, a factor of ten. And then another factor of ten as we
4 go to .999. So all I'm saying is the odds ratio by itself
5 can create a misleading impression if you don't also see
6 these numbers.
7 Q. Is there something about the mathematical
8 characteristic of the odds ratio that causes this, I mean,
9 is that the reason?
10 A. The basic problem is that an odds ratio requires
11 division. And if one of the probabilities is either near
12 one or near zero, we encounter something called division by
13 zero which is prohibited, mathematically. We can't have a
14 fraction that has zero and nine.
15 Q. And so what's the results of that?
16 A. And so as the denominator goes towards zero, the
17 fraction increases without bound to incredibly large
18 numbers. If we keep adding nines, this thing keeps going
19 up and up and up.
20 Q. And does the same pattern happen when you're talking
21 about small probabilities at the other end?
22 A. Exactly the same pattern happens, so, for example, if,
23 I just switch it around. If group one had a probability of
24 admission of .10, and group two had a probability of .10,
25 the odds ratio would be eleven.
69
1/19/01 - BENCH TRIAL - VOLUME IV
1 If I went from, again, group one, .0 to group two
2 .001, 111, .10 to .0001, 1,000, 111. So again, group one,
3 ten percent chance of getting in, group two, very small
4 .10, very small, very small. Ten percentage point
5 difference leads to very, very different odds ratios.
6 Q. Do you have an example of a situation in which two
7 people might have similar probabilities of something
8 happening, but very different odds or a real world example?
9 A. Yes, actually, I did think of one. It actually
10 involved the lottery. Suppose that, you know -- I get
11 excited about the lottery and I buy a lottery ticket. And
12 you say, well I'm going to outdo you, I'm going to buy
13 fifty lottery tickets.
14 So what would happen is your odds would be roughly
15 fifty times, mine. But yet both of us would have near zero
16 probability of winning the lottery. I mean, it's wise,
17 you'd say, I'm going to be really smart and go buy
18 thousands of tickets to the lottery. Everybody would be
19 buying. Of course they are, but.
20 THE COURT: Actually this week it's fifty-nine
21 million. There's a sign on my way home. Every time I keep
22 looking.
23 A. They're doing it. They're rapidly increasing their
24 odds, but what they don't know is their probability is
25 staying right almost exactly at zero.
70
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. All right. Okay. If you could take the stand.
2 Professor Raudenbush, in your view does this pattern that
3 you've just described to us examples have any relevance to
4 the data we have in this case?
5 A. They do. There are combinations of grade point
6 average and LSAT where the probability of admission of
7 anyone who applies to the law school is extremely high. I
8 mean, people who have near A averages who are up in the
9 upper 160's or 170's on their LSAT have an extremely high
10 probability of admission.
11 Of course in the data what we see is that the
12 proportions are something like 1.0 for minority applicants,
13 and something in the .9 range, or in a very high range for
14 majority applicants. And so in that sense, the examples
15 that I was presenting were not unusual. And something
16 similar can also, and does appear at the lower end of
17 people who have fewer qualifications where the differences
18 may be small in probability terms, but the odds ratios may
19 be big.
20 Q. With that background in mind, I'd like to ask some
21 questions about the cell-by-cell analysis, that Dr. Larntz
22 conducted.
23 A. Okay.
24 Q. Just, let's start with a general question. What's
25 your opinion about the of the appropriateness or the
71
1/19/01 - BENCH TRIAL - VOLUME IV
1 validity of that approach?
2 A. Well the problem, well, one of the problems with that
3 approach is that it requires that an odds ratio be
4 computable for every single one of the hundred plus cells
5 that appear in any year in Professor Larntz' reports. And
6 since the odds ratio is not computable in a number of
7 cases, what this leads to is a discarding of data in those
8 cases where there can't, where no odds ratios is
9 computable. And this ends up discarding considerable
10 evidence that are relevant to how the university is
11 handling the admissions decisions.
12 Q. And I believe we had some examples?
13 A. Yes.
14 Q. Of those situations?
15 A. We do.
16 Q. I think this is Exhibit 192. If you'd put that up.
17 Maybe, David, if you could put the easel back where it was.
18 Can you read it from there?
19 A. Yeah, I can see the numbers from there.
20 Q. Okay. There's very small, actually.
21 A. I'll --
22 Q. I think I need to come closer.
23 A. Okay. All right.
24 MR. DELERY: If that's all right, Your Honor.
25 THE COURT: Of course.
72
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. So I guess let me first just ask, I take it this page
2 here on the left is a little image of a page from one of
3 Dr. Larntz' reports?
4 A. Yes. That's page six of six from the March 20, 2000
5 report. And we selected that page. It was just convenient
6 because it had three examples that I wanted to say
7 something about, because it has a bearing on what we're
8 discussing, and they all came from the same page.
9 And the first example, actually, the first two
10 examples involve cases in which the admissions process
11 treated people the same, in terms, they had the same
12 admission decision regardless of minority status. So in
13 the first cases, and we're looking here at, at students who
14 have relatively low grade point average. It's down 2.25 to
15 2.49, but relatively high LSAT's, 161 to 163. There was
16 one minority applicant in that, in, who had those
17 characteristics. And that person was rejected. There were
18 two majority applicants and they were rejected because both
19 people were rejected. Of course, what we know is they both
20 had the same admissions decision. There was no different
21 decision for the minority and majority applicants. But
22 because none of them were admitted, we can't compute the
23 odds ratio. So if you've developed a statistical approach
24 that requires cell-by-cell computation of odds ratios, you
25 can't compute the odds ratio.
73
1/19/01 - BENCH TRIAL - VOLUME IV
1 Basically what happens is you have to discard
2 their cell. But when you discard this cell, you're
3 discarding information that's relevant to the decision of,
4 it's relevant to the decision made by the admissions
5 committee. That is, essentially, you're waiting to see
6 what the admissions committee decides.
7 And if they make it a certain decision, which in
8 this case is treating everybody the same by rejecting them,
9 discard the data. If the admissions decision had been
10 different, if, if someone had been admitted, then the cell,
11 the data would have gone into the analysis. So that means
12 the data goes into the analysis conditional on the decision
13 of the university.
14 If the university makes a decision to treat
15 everybody the same, we throw the data out. If the
16 university decides to treat them differently, the data go
17 in and we -- we don't like that situation in statistical
18 analysis. This, we don't wait to find the outcomes of the
19 data and then decide whether to use the data. We decide
20 what data we're going to use prior to, to, to investigating
21 the outcomes, or without any attention paid to what the
22 outcomes are that we're trying to discuss.
23 Q. Okay. And what about the second cell here?
24 A. Well, the second cell is another example of the same
25 thing but it's at the upper end of the distribution.
74
1/19/01 - BENCH TRIAL - VOLUME IV
1 In this case we have people whose grades are 3.75
2 and above. This is very high. These people are getting
3 basically A's, maybe a few A minuses. Their LSATs are also
4 very high. They're 167 to 169, which I think is very high
5 up in the percentiles of that distribution so these are, in
6 terms of just grades and test scores this is a very able
7 group of applicants.
8 In 1999 there were two minority applicants. They
9 were both admitted. There were 106 majority applicants.
10 They were all admitted. So you look at those data, and I
11 think reasonable people would say, did race play a factor
12 in the decision for those people. And the answer seems to
13 be no. They had very high grades and very high test
14 scores. They all had the same decisions. The decisions
15 weren't different. However, can't compute the odds ratio,
16 throw out the data.
17 Q. Well, let me ask you about that, because Dr. Larntz
18 said that these cells don't have comparative information in
19 them, as I understand it, and so a principle or fair
20 comparison should mean that you would discard them. You
21 would look at only cells where you have different results.
22 Do you disagree with that?
23 A. I strongly disagree with that, and I'll try to explain
24 why. We only know after the fact that these people had the
25 same treatment. To then say, well, because they had the
75
1/19/01 - BENCH TRIAL - VOLUME IV
1 same treatment we're going to throw them out, no, you can't
2 do that. The admissions decision could have gone the other
3 way. And that's what we have to think about in statistics.
4 THE COURT: Everything in every cell is after the
5 fact?
6 A. Right. But we don't use or not use data, depending on
7 what we see in terms of who was admitted. The principle,
8 the actual principle, statistical principle is you use all
9 of the information in the data.
10 THE COURT: But all of the information in the data
11 that he says, at least, that he wanted to use, and that was
12 necessary was after the fact. He didn't combine after the
13 fact with before the fact.
14 A. He decided which, which parts of the data to use after
15 he saw, based on the results of the admissions decision.
16 We don't, we don't decide, well, I'm not going the analyze
17 these because these people were admitted or rejected. We
18 don't. That's not the way we do it. I mean, these are
19 results that are discriminate with his hypothesis.
20 THE COURT: You were here for his testimony?
21 A. Yeah.
22 THE COURT: He said as to these cells there was
23 nothing to analyze?
24 A. And that's not true. That's simply not true. I did
25 the analysis myself. I used every scrap of data that there
76
1/19/01 - BENCH TRIAL - VOLUME IV
1 was. We can analyze all of it. We must analyze all of the
2 data.
3 THE COURT: So you disagree with him?
4 A. I strongly disagree. And I say that the reason he
5 discarded those data was because he was committed to a
6 cell-by-cell computation of odds ratios, and they can't be
7 computed.
8 Q. So in other words, a different methodology would have
9 allowed all of the data to be include, is that right?
10 A. That is absolutely right.
11 THE COURT: Did you do that?
12 A. I did.
13 THE COURT: I expect we'll know your results?
14 A. Yes. Actually all of the analyses I've reported so
15 far never, we never selected cases for the analysis as a
16 function of whether people were admitted or not. We always
17 analyzed whatever data came to us.
18 Q. Now, as a statistician, if you had selected a
19 methodology and then you saw that it was leading to the
20 exclusion of a number of cases from the data, would that
21 cause you to think about your methodology in any way?
22 A. It would very much cause me concern. And let me,
23 maybe I can explain a little bit in a very simple
24 straight-forward way how this could be so consequential.
25 Suppose another statistician came along and had
77
1/19/01 - BENCH TRIAL - VOLUME IV
1 never seen Larntz, the, the report of Professor Larntz, but
2 had the database and decided to create cells. But suppose
3 that this statistician decided to create bigger cells.
4 Let's say, let's take everybody from, instead of
5 just LSAT from 161 to 163, I'm going to create larger
6 cells. I'm going to take everybody from 161 to 165. So
7 you'd have bigger sample sizes. What would inevitably
8 happen is that you'd have a larger fraction of the cells
9 where you could compute the odds ratios.
10 And, in fact, you could define the cells big
11 enough so that you could compute an odds ratio for every
12 cell. So what would happen, statistician No. 2 would come
13 along and define the cells somewhat differently, using the
14 same methodology, would throw away different cases, fewer
15 cases and get different results, quite different results,
16 in fact.
17 Statistician No. 3 comes along and says, I don't
18 like really, these cells, they're too big. I like really
19 small cells. I'm going to define cells that only go LSAT
20 from 161 to 162 because I want to equate people really
21 closely to every LSAT point. So I'm going to have, and
22 that would create basically roughly twice as many cells.
23 What would happen to statistician No. 3,
24 statistician No. 3 would see a lot more small cells where
25 nobody was admitted or everybody was admitted, and would
78
1/19/01 - BENCH TRIAL - VOLUME IV
1 have many fewer computable odds ratios than Professor
2 Larntz and would throw out a great deal more of the
3 information.
4 So now what we have is using this methodology of
5 constructing cells and then for each cell computing odds
6 ratios, we have three statisticians using the same
7 methodology but they define the cells differently. They
8 never talked about it. They just define them differently.
9 And they're analyzing different data sets with different
10 subsets of cases that have different outcomes and they're
11 coming up with different results. That is not what we,
12 what we aim for in statistical practice.
13 Q. And in your opinion, does that fact, the fact that
14 different size cells would, would lead to different
15 results, does that tell you anything about the
16 appropriateness of the cell by cell approach, in general?
17 A. Exactly. Because it's the cell-by-cell computation of
18 the odds ratios that, that, which involves division, which
19 we can't divide by zero, or, or to paraphrase, Professor
20 Larntz, we can't divide infinity by infinity or zero by
21 zero. And so if you're committed to that strategy, you
22 have to discard data from cells that, where you can't
23 compute the odds ratio, but it turns out that the cells
24 you're discarding are the ones where people are being
25 treated the same; in many cases the same, as a function of
79
1/19/01 - BENCH TRIAL - VOLUME IV
1 ethnicity.
2 Either they're being rejected or they're all being
3 accepted. The key thing we have to keep in mind as a
4 statistician is it could have gone the other way. In a
5 different world with a different policy, some of these
6 people would have been rejected, and we have to, our models
7 have to anticipate that the world might be telling us a
8 different story.
9 Q. So would you choose a cell-by-cell approach?
10 A. No, I would not.
11 Q. Why don't we look at the third cell that you have here
12 on this chart. Tell us what you find significant about
13 that one?
14 A. Well, this cell, actually exemplifies something we
15 discussed earlier and how an odds ratio by itself can be
16 misleading.
17 What we have here are candidates who have grades
18 in the sort of B plus range, but very high LSATs. And
19 there was one minority applicant, there was one minority
20 applicant and one minority admit. So one person applied
21 and was admitted. There were 75 majority applicants and 73
22 admits. So in terms of proportions, 100 percent of the
23 minority applicants were admitted, and 907 percent of the
24 majority applicants were admitted.
25 When we compute the odds ratio, we come up with an
80
1/19/01 - BENCH TRIAL - VOLUME IV
1 infinite odds. We can't really compute that number. We
2 can't really divide by zero. But if labeled infinite, and
3 it conveys the impression that minority applicants were
4 much more likely to be admitted, and yet if we look at the
5 proportions, it's 1.0 versus .97. Those look very similar
6 and I think most people would say on balance looking at
7 that cell, applicants were treated similarly.
8 Q. Would you call an odds ratio for a cell that turns out
9 to be infinity a calculable odds ratio?
10 A. No, I would not because it involves division by zero
11 which we can't do. The computers won't let us do it.
12 Q. Ms. Massie, the other day, or yesterday, I think,
13 asked Dr. Larntz whether infinity is an irrational number
14 or an imaginary number. What is it?
15 A. Well, actually, it's not a number. In fact, if you
16 try to, if, in many computer programs, if you try to divide
17 by zero, it will print out, n-a-n, not a number.
18 Q. All right. Given the, well, what do you take away
19 from the fact that the cell-by-cell approach of Dr. Larntz
20 generated odds ratios of infinity in this way?
21 A. I take away that, that analyzing many, many, many
22 small subsets of data, using this method is not the right
23 way to go, and it will lead to distortions. It will lead,
24 in fact, to an exaggerated estimate of the association
25 between minority status and ethnicity, both in terms of
81
1/19/01 - BENCH TRIAL - VOLUME IV
1 which data are discarded and which data are analyzed, and
2 in terms of these unstable odds ratios which veer to become
3 increasingly large, depending on the denominator and in the
4 division.
5 Q. Okay. Is it fair to say that the pool sizes, in other
6 words, the number of applicants in a number of these cells
7 are quite small?
8 A. Yes. That's correct.
9 Q. Do you find that fact significant in any way in
10 evaluating the appropriateness of this approach?
11 A. It's because they're small that so much of the data
12 are discarded using this approach. And that's really the
13 key. That's a key issue. And it's also because they're
14 small that the odds ratios become so unstable.
15 Q. Now, it is the case, I mean we have put up here
16 examples of cells in which the minority and majority
17 applicants were treated quite similarly?
18 A. Right.
19 Q. That's what we've selected. It's also the case, isn't
20 it, that there are cells where the probabilities of
21 admission are quite different, or the proportions of
22 admission are quite different?
23 A. That's true.
24 Q. Where large proportions of minority students are
25 admitted and very small proportion of the majority students
82
1/19/01 - BENCH TRIAL - VOLUME IV
1 are admitted?
2 A. That's correct.
3 Q. Have you looked at those cells in any way?
4 A. We have. We've looked at those cells and I think we
5 actually have a display of one of them that will reveal
6 some of the features of those cells.
7 Q. Okay. This is Exhibit 193, I believe. So am I right,
8 Professor Raudenbush, that this sample cell is taken from a
9 page of Dr. Larntz' report, March 20 of 2000, the same
10 report as the page we just saw?
11 A. Yes, and it's page five of six.
12 Q. Okay. Well, why don't you tell us what the cell
13 shows, and then what you take away from it?
14 A. Okay. The cell is, includes people whose grades were
15 3.50 to 3.74, which is in the B plus to A minus range.
16 Their LSAT scores are 156 to 158, which is, which is
17 comparatively high. It's in the seventy-first to
18 seventy-eighth percentile so they're pretty high up in the
19 percentiles of the LSAT. They were non-residents. There
20 were seven minority applicants, and of those six were
21 admitted, six out of seven.
22 There were 73 majority applicants, and of those
23 one was admitted. So one out of 73, obviously, six out of
24 seven looks quite different from one out of 73, and
25 Professor Larntz' computed an odds ratio of 432 and a
83
1/19/01 - BENCH TRIAL - VOLUME IV
1 probability value of point, less .0001.
2 Q. And what does that probability number stand for?
3 A. That is a test of the, what we call the null
4 hypothesis. The null hypothesis is that in this cell
5 there's no association between race and admissions. And so
6 if the null hypothesis were true, there's no association,
7 how likely is it that we would see results like this, and
8 the answer is not very likely. We, therefore, reject the
9 null hypothesis and infer there is a statistical
10 association between race and admissions in this cell.
11 Q. Okay. And that's what, that's also what the odds
12 ratio indicates, is that right?
13 A. The odds ratio is, it's hard to know what the odds
14 ratio really indicates by itself, 432. I mean we've seen
15 some cases where a number like that might not mean much at
16 all, but in some cases it might mean a lot.
17 In this case we can see that six out of seven is
18 different from one out of 73 and that's, in proportion
19 terms, those are pretty big differences. Of course we
20 don't know how big that difference is, we can't put a good
21 confidence interval because the sample size of minority
22 applicants is small.
23 What I mean is it's hard for us to say just how
24 big the effect is. We know there's affect. But to really
25 bound it is difficult because of the small size of the
84
1/19/01 - BENCH TRIAL - VOLUME IV
1 sample of the minority applicants.
2 Q. Have you looked at another way to think about the
3 effect of race in this cell?
4 A. Yes. What we've done with this cell is just to do a
5 little mini-simulation, just to do the causal analysis that
6 we did earlier, but only with this cell, and it's really
7 pretty straight forward. In this cell, there were eighty
8 applicants overall and seven were admitted. So the common
9 probability of admission observed here is the seven divided
10 by eighty, which is .0875. So here's a very simple way of
11 simulating a race-blind policy.
12 Suppose that common probability of admission were
13 applied to the majority applicants and the minority
14 applicants. How many admits would we then expect under the
15 race-blind policy. So we multiple .0875 by 73, and we get
16 six. We round off, we can't admit half a person. So we
17 have to round off six majority admits and then the same for
18 minorities. We take the common probability of admission.
19 .0875, multiple by seven, and we get one minority admit.
20 So here's kind of a real simple way in which the
21 simulation works. What we saw in reality seven minority
22 admits, I'm sorry, seven minority applicants, six admits,
23 and one admit for majority. Under the race-blind policy it
24 would switch, six majority admits and only one minority
25 admit.
85
1/19/01 - BENCH TRIAL - VOLUME IV
1 We can then compute the change in probabilities
2 for the majority students under the race-blind policy.
3 Under the current policy, one out of 73 was admitted.
4 That's 1.4 percent. Race-blind policy, six out of 73.
5 That's 8.2 percent.
6 So what that gauges is the causal, is -- it's an
7 estimate with uncertainty. But it's an estimate of the
8 causal impact of changing to a race-blind policy for the
9 majority students. Their probability of admission goes up
10 from 1.4 percent to 8.2 percent, which is definitely an
11 increase. It's an increase of about 7 percentage points.
12 Under either policy their probability of admission is less
13 than one in ten. And that's a way of quantifying what's
14 going on in this cell, and I guess it just shows, this is
15 kind of how we're quantifying what's happening in the cell,
16 as opposed to simply using a number of an odds ratio 432.
17 Q. Do you think that 432 odds ratio quantifies how much
18 race has been considered in the admission process for the
19 applicants in this cell?
20 A. No, I don't have.
21 Q. And why is that?
22 A. Well, the idea, to make an inference about the role of
23 race, the extent to which race was taken into account in
24 admissions, we would have to infer or assume that everyone
25 in this cell is identical, in terms of their other
86
1/19/01 - BENCH TRIAL - VOLUME IV
1 credentials. If that quantifies the impact, we're assuming
2 that these people are the same.
3 Actually, let me back up. We're assuming that
4 other factors are unrelated to grade point average and test
5 score. But the basic idea is we're assuming that these
6 people are very similar in terms of their credentials.
7 Q. Well, Dr. Larntz, as I understood it, said that his
8 general approach was to try to identify similar students,
9 and then look at the relative?
10 A. Right.
11 Q. Relative odds of their acceptance.
12 THE COURT: Using similarly as to those who
13 factors?
14 MR. DELERY: Right.
15 THE COURT: Grade point and the exam.
16 MR. DELERY: Right.
17 Q. Well let me ask you --
18 A. Well actually there were a couple analyses; one
19 controlled for just grade point and LSAT, another
20 controlled for residents, gender, fee waiver, several other
21 factors.
22 THE COURT: Yeah. That was a separate analysis?
23 A. That was another analysis, yeah.
24 THE COURT: But his main premise that he used.
25 A. Right.
87
1/19/01 - BENCH TRIAL - VOLUME IV
1 THE COURT: Only the two that, I think we're on
2 the same wave length?
3 A. Yes, that's correct.
4 THE COURT: Only used those two and he explained
5 the reasons?
6 A. And those are the big ones in terms of predictive
7 power, right.
8 Q. Let me ask you this, if we had data for all of the
9 factors that are considered by the admissions process, and
10 we know that we don't, as we discussed earlier, but
11 assuming, hypothetically, that we had statistical data on
12 all of those factors, so that you could identify students
13 who were exactly the same.
14 A. Right.
15 Q. On all of the factors that the admissions office
16 considers, except that they differed by whether they were a
17 minority or not?
18 A. Yes.
19 Q. What would happen then to the odds ratio?
20 A. In that case, it would have to be infinite.
21 Q. And why is that?
22 A. It would have to be infinite for this reason. Let's
23 just say there are ten factors that can, that can account
24 for admissions. And we have people who are identical on
25 all nine, nine of those ten, but they're different by the
88
1/19/01 - BENCH TRIAL - VOLUME IV
1 last factor. Then that last factor must determine the, any
2 outcomes that weren't already determined by the previous
3 nine. It just, logically, has to be true.
4 Any admissions decisions that were not dictated by
5 the nine would have to be then made by that last factor,
6 and so because it's the only thing that can explain what's
7 left, the odds ratio would have to go to infinity.
8 Q. So in my hypothetical example, would race have to be
9 taken into account a lot to yield the infinity odds ratio?
10 A. And the answer is no. It would not have to be taken
11 into account a lot. If it were taken into account a little
12 or a lot, if it's the last, the only last thing that could
13 be effecting this decision, you would still have an
14 infinite odds ratio.
15 Q. Do you have an example that could illustrate that in
16 some way?
17 A. Yeah. I tried to think of something that would make
18 this point sort of clear. If I have a scale with two sides
19 on it and it's in a balance and I put something on that, on
20 one side of that scale and I see it go, one side go down, I
21 can't infer how heavy the thing was that I put on that
22 scale.
23 I mean, I could have had, there could have been
24 one pound on this side and one pound on this side and I
25 added a pound to this side and it went down. I could have
89
1/19/01 - BENCH TRIAL - VOLUME IV
1 had a thousand pounds on this side and a thousand pounds on
2 this side. I could have put an ounce on this side. It
3 would have gone down, so knowing that the scale went down
4 decisively, cannot tell us how big the weight was that made
5 it go down.
6 THE COURT: Can you tell us how big it wasn't,
7 though? You can tell us it wasn't a feather or two
8 feathers or six feathers?
9 A. Well, theoretically, any weight, if there was an exact
10 tie, any weight would have made the scale, even an ounce.
11 THE COURT: I'm talking about degrees?
12 A. But what I'm saying is if these are in actual, in
13 absolute balance, these two sides, and any weight
14 whatsoever, no matter how small is put on this side, it has
15 to go down. So the only thing we can infer is that there's
16 more weight on this, that something was put on the side.
17 We know that something happened, that this last thing was
18 taken into account, but we can't tell how much we put.
19 THE COURT: That's right, but what I'm saying is
20 your analogy if they're both equal, this one goes down, you
21 put something on this one and it goes down just a little
22 bit, you know, you just put a little bit on if it goes down
23 a little bit more go.
24 A. Well, if they're really imbalanced, it will go down,
25 it will go all the way down.
90
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. Can I say, to bring us back to what we're talking
2 about?
3 A. Yes, a scale. A balance beam is maybe a better
4 analogy as to what I'm saying.
5 THE COURT: I don't know. It's funny, I've seen
6 them a hundred times in court. I've never used one?
7 A. Let's say like a teeter-totter. I don't do those any
8 more, but, you know, it's kind of sitting there and it
9 might just be sitting there and, basically, if someone sits
10 on one end it goes down to the ground.
11 THE COURT: I got you.
12 A. Now, that could be a little child or it could be, you
13 know, a huge football player. It would still go down to
14 the ground. The fact that it's on the ground doesn't tell
15 us the size of the person who's sitting on that
16 teeter-totter.
17 Q. In the admissions decision we're talking about a yes,
18 no decision?
19 A. Yes.
20 Q. Is that right?
21 A. Right.
22 Q. Right. Not one of degree?
23 A. Right.
24 Q. If you come back to the cell here that we were looking
25 at, I think Dr. Larntz would say that he could infer
91
1/19/01 - BENCH TRIAL - VOLUME IV
1 something about the extent to which race was taken into
2 account by how big that odds ratio number is, that because
3 it's so big, that must mean that race was a big factor in
4 the decisions?
5 A. Right.
6 Q. Do you disagree with that?
7 A. I disagree with that for the reasons we've just
8 described. Knowing that the proportion went up for one
9 group doesn't tell, has no information about the extent to
10 which the people making the admissions decisions were
11 relying on that factor. It, the analysis suggests that
12 there is, that it is being taken into account, and the idea
13 that it's not, that it's absolutely irrelevant is the null
14 hypothesis. We rejected that, but the extent to which it's
15 being taken into account you can't determine from this
16 analysis.
17 Q. I'd like to turn now from the cell-by-cell approach
18 that we've been talking about to the composite odds ratios
19 that Dr. Larntz generated?
20 A. Okay.
21 Q. Do you have an opinion concerning the meaningfulness
22 of those composite or global odds ratios?
23 A. I do.
24 Q. And what is that?
25 A. And that is that I, I do not view them as a valid
92
1/19/01 - BENCH TRIAL - VOLUME IV
1 assessment of the association between race and admissions,
2 given test scores and GPA, which is a narrow way of
3 defining what the analysis was intending to estimate.
4 Q. And in your view, does the choices that Dr. Larntz
5 made about the cells that we discussed earlier, do those
6 choices have any implications for your evaluation of the
7 composite odds ratios?
8 A. They do. And first let me just mention again the, the
9 question of making a methodology decision that then
10 influences which outcome data we throw away and which
11 outcome data we pay attention to.
12 In general, using logistic regression we do not
13 need to discard any cases from this, from -- there are,
14 there are no cases, no people whose data needs to be
15 rejected. And that's, I mean, in a nutshell, we can handle
16 that.
17 What Professor Larntz did was to construct for
18 every cell in the matrix that had any data at all a
19 predictor variable. Well, I should say, what he did was
20 construct for every cell in the matrix that had what he
21 defines as comparative information. He gave a very clear
22 definition of that yesterday. He defined for those cells a
23 predictor variable. That means that in his logistic
24 regression model he had approximately 100 predictor
25 variables, one for each cell that had the kind of data that
93
1/19/01 - BENCH TRIAL - VOLUME IV
1 he found to be useful, or one that could be used in that
2 context, one that had the features that he described which,
3 you know, was a good description of what it was. Having
4 the 100 predictor variables, one for each cell, requires
5 that you discard cells of the type we looked at that were
6 just also discarded cell-by-cell analysis. And, and so
7 that was a decision to construct the many, many, many
8 predictor variables, one for each cell that had the same
9 consequence it did in the cell-by-cell analysis.
10 Q. Okay. What about the cells that would generate on a
11 cell-by-cell basis the infinity odds ratios that Dr. Larntz
12 found? What happened to those cells in his analysis?
13 A. Yeah. Well, those cells, and he pointed this out,
14 it's not as if we'd be averaging infinity with three other,
15 with a hundred other numbers.
16 But what we would tend to be averaging in many
17 cases, or combining, would be numbers that are very, very
18 high, as a function of the number of predictor variables in
19 the model, and the number and the small size of those
20 samples. And when you create a composite across a hundred
21 numbers, many of which are very high for those reasons, you
22 get an unstable composite estimator, and that's what we see
23 in this case. When I say unstable, I mean it varies from
24 year to year much more than we would expect, given the data
25 at hand.
94
1/19/01 - BENCH TRIAL - VOLUME IV
1 Q. I'd like to come back to the instability point.
2 A. Right.
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
95
1 BY MR. DELERY:
2 Q. Before we get to the instability point, I would like
3 to talk about the assumptions issues that you
4 mentioned a few minutes ago.
5 A. Right.
6 Q. What were the assumptions that Dr. Larntz made that
7 you found significant, and what do they tell you
8 about his analysis?
9 A. Well, there were two really central assumptions in
10 this kind of analysis. When what you're trying to
11 do is characterize the difference between two
12 groups, or an odds ratio that expresses this
13 difference controlling for a large number of other
14 factors.
15 The assumption is that the size of
16 the difference is, or in this case the size of the
17 odds ratio, is invariant across all of the cells of
18 the matrix.
19 That is literally all of the cells
20 have the same true odds ratio. That's a very
21 important assumption for this analysis.
22 And I actually did some analysis to
23 check that assumption, and found that it was easily
24 rejected. Indeed, the size of the odds ratio varies
25 significantly across the cells of the matrix.
GRUTTER -vs- BOLLINGER, ET AL
96
1 Q. And by the matrix, what do you mean?
2 A. The matrix meaning the GPA by LSAT grid, which is
3 what Professor Larntz was using in the analysis.
4 And this is a case where the higher
5 arc of the linear model actually is very useful,
6 because we have many, many small subsets of data.
7 It's almost like we have children in classrooms and
8 we actually have applicants and cells.
9 Q. And that's what your book was about?
10 A. That's what my book was about.
11 Q. That's what we're talking about.
12 A. With how you handle data where we have many, many
13 small subsets of data. How do we combine the
14 information in such a way, that we are not required
15 to discard the information.
16 And what we do is we have the
17 following conception. That every cell has its own
18 true odds ratio and that they have variability
19 across the cells. They randomly vary across the
20 cells.
21 The beauty of that is we only have to
22 estimate this one parameter, how much variability is
23 there across the cells. We don't have to estimate
24 each cells odds ratio.
25 And when we do that analysis what we
GRUTTER -vs- BOLLINGER, ET AL
97
1 find is that there is very substantial variability
2 across the cells of the matrix in terms of the odds
3 ratio.
4 Now, this is something that you can
5 see by looking at the data, you can actually look at
6 the data, you can see that in the upper ends where
7 people have very high grades and test scores,
8 they're being treated very similarly.
9 And in the more middle ranges like
10 the cell we are now looking at still, I guess, the
11 one that had the odds ratio here, that the odds
12 ratios become quite large.
13 So, they do vary or the cells of the
14 matrix, and that contradicts an assumption that's
15 very important. And means that we can't
16 characterize the association between race and
17 admissions with a single odds ratio.
18 Q. And just so we're clear, could you sort of expand a
19 little bit on why it's significant that this
20 assumption is wrong. What does that mean about the
21 usefulness of Dr. Larntz's odds ratio?
22 A. Well, substantively one feature of it is that if the
23 actual difference in probabilities, or the odds
24 ratio is bigger in some areas than others, that's a
25 very different story then saying every person who
GRUTTER -vs- BOLLINGER, ET AL
98
1 applies to the law school is going to be subjected
2 to this odds ratio, that's one thing. But it has
3 certain technical results.
4 If, in fact, the odds ratios vary
5 across the cells and you think they don't, what
6 happens is the standard error of your estimates
7 becomes too small. And any confidence intervals and
8 test of significance become questionable.
9 Q. In practical terms what does that mean about how you
10 interpret the odds ratio over the cells?
11 A. In practical terms what that means is that we can't
12 bound the size of the quantity we're trying to
13 estimate. We can't put upper and lower bounds on
14 the size of the relationship that we're trying to
15 estimate.
16 Q. So it could be larger or it could be smaller than
17 the results that Dr. Larntz report?
18 A. It could be larger or smaller. Generally when we
19 report a result, a number, like if I say the odds
20 ratio is 432, I typically would say, well, but what
21 are the upper and lower bounds of the possible odds
22 ratios that we might have gotten, because we don't
23 believe it's actual exactly 432.
24 To do that we need a standard error
25 that's reliable, and we can't get a standard error
GRUTTER -vs- BOLLINGER, ET AL
99
1 that's reliable if that assumption fails.
2 Q. And as a statistician if you can't get a standard
3 error to put a bounds around the number like 432,
4 what does that tell you about how much weight you
5 can put on a number like that?
6 A. Well, it tells that, basically it tells you that the
7 odds ratio is greater than one, it's significantly
8 greater than one. But it doesn't quantify the
9 extent to which it's greater than one.
10 Q. So, in other words, is it fair to say you can reject
11 and nullify hypothesis, but can't quantify the
12 extent beyond that?
13 A. That's right.
14 Q. Now, we talked earlier when we talked about your
15 models and also about Dr. Larntz's, that you both
16 had to assume that factors that you couldn't put in
17 your models were unrelated to the factors that were
18 in your models?
19 A. Correct.
20 Q. In your review of Dr. Larntz's work, does that
21 assumption mean anything about the significance of
22 his results?
23 A. Well, anytime we estimate a logistic regression
24 equation, we are almost always required to make this
25 assumption, because it's almost always true that
GRUTTER -vs- BOLLINGER, ET AL
100
1 there are a lot of things we don't know that are
2 important.
3 And in that regard as I mentioned, my
4 logistic regressions are vulnerable to the
5 criticism, we've discussed that.
6 The problem here was that we had no
7 way of checking to see the extent to which the
8 failure of that assumption might have affected the
9 results. We can't put upper or lower bounds.
10 We know that the assumption is false,
11 at least to some degree. It may be trivial, it may
12 be a large degree, but we can't assess the extent to
13 which the falsehood of the assumption might have
14 affected the results.
15 Q. Why can't you or Dr. Larntz put a bound on his
16 numbers in the same way that you did on yours?
17 A. Well, the method that I'm using--well, you could
18 actually create a bound, but it would be so wide it
19 would go from zero to infinity. I mean it would be
20 extremely wide.
21 There's just no way in this context
22 to have a strong sense of what the upper and lower
23 bounds are.
24 What we would want minimally would be
25 a confidence interval, the validity of which would
GRUTTER -vs- BOLLINGER, ET AL
101
1 still be contingent upon the assumptions. But, at
2 least, that would be a way of bounding the quantity.
3 But we don't have that here.
4 Q. You had mentioned an instability point earlier and I
5 deferred you on that, I would like to return to it.
6 What is your point about the
7 instability, I think you called it, of Dr. Larntz's
8 odds ratio?
9 A. When we discussed some of the sources of it, the
10 cell by cell analysis, the instability of the odds
11 ratio itself, the choices as to which data are used
12 and not used could possibly feed into it.
13 What I did was I simply looked across
14 the years at the results from Professor Larntz's
15 reports and I look at the odds ratios. I think we
16 have an exhibit.
17 Q. Okay. Why don't we turn to that, Exhibit 194.
18 Looking first at the left side of the chart, if we
19 could. Am I right that these are odds ratios that
20 were taken from Dr. Larntz's various reports?
21 A. That's correct.
22 Q. Okay. And there are three columns here for Model
23 One, Model Two and Model Three, what did those mean?
24 A. Well, as Professor Larntz presented models, he
25 presented results from models that controlled only
GRUTTER -vs- BOLLINGER, ET AL
102
1 for grades and test scores.
2 He had a second model that in
3 addition controlled for Michigan residents, gender,
4 fee, fee status waiver. And also numerical
5 discrepancies in GPAs, it was more elaborate model.
6 And then the third model was one
7 where we used the selection index as opposed to the
8 GPA and LSAT.
9 Q. And that was a third model that we didn't hear about
10 during his testimony, is that right?
11 A. Right, correct.
12 Q. So then you have listed the odds ratio from his
13 reports under the three models?
14 A. That's true.
15 Q. What do you conclude based on the pattern of numbers
16 here across the years?
17 A. Well, when I look at the numbers across the years
18 within a model, we can just take model one. There
19 is really very large variability in these numbers.
20 For example, in 1997 the odds ratio
21 for African Americans was 53.9, whereas in 2000 it
22 was 443.26.
23 Now, if we took those numbers
24 literally, it would imply that the relative
25 advantage of African Americans was basically nine
GRUTTER -vs- BOLLINGER, ET AL
103
1 times as great in 2000 as it was in 1997.
2 Which would imply a very big change
3 in the policy. It would imply that the data would
4 look different.
5 And, of course, looking at these
6 numbers and knowing that the odds ratio itself can
7 be unstable, my first impulse was to assume, and I
8 think this was Professor Larntz's, that these
9 numbers were varying by chance. They're big odds
10 ratios. That we saw that odds ratios can become
11 unstable.
12 However, I took another step which
13 was to look at the standard errors of the
14 differences between any pair of odds ratios. These
15 standard errors are basically in the report that
16 Professor Larntz--in his report.
17 Q. You can derive them from information?
18 A. The standard errors from each year are derivable
19 from the report. And we can then easily compute a
20 standard error for the difference between any two
21 odds ratios.
22 And what I found was--
23 Q. Let me just interrupt for a second.
24 A. Sure.
25 Q. That's a standard statistical technique that you
GRUTTER -vs- BOLLINGER, ET AL
104
1 performed?
2 A. Yes.
3 Q. Yes.
4 A. Anytime we want to know how big is the difference
5 between two numbers, we compute what we call a
6 standard error of the difference, and how many
7 standard errors are they apart. Professor Larntz
8 referred to these as standard deviations.
9 Q. Okay.
10 A. What I found was that the standard deviations in
11 2000 were--I'm sorry, the odds ratio 443 in 2000 was
12 eleven standard deviations bigger than the odds
13 ratio in 1997.
14 Q. And, in your opinion, what's the significance of
15 that fact?
16 A. Well, that kind of a difference could stem from a
17 difference in the policy. It would have to be a
18 very big difference, leading to a very big
19 difference in how the basic data looked. Or it
20 would have to simply be a function of the
21 methodology.
22 And so if you look at the law
23 school's policy, the same policy was in effect from
24 1992 to the present. There's no reason to believe
25 that it dramatically changed, that there was
GRUTTER -vs- BOLLINGER, ET AL
105
1 tremendously increased weight put on African
2 American admissions.
3 When we look at the data, what we
4 have on the right panel, is the percentage of people
5 admitted, African Americans versus Caucasian, those
6 number are stable, they're similar. They're not
7 that different.
8 In '97 it was 34 percent for African
9 Americans, 39 percent Caucasian. 2000 it was 36
10 versus 41, those are very similar.
11 And so my conclusion is, that the
12 instability in the result stems not from changes in
13 the policy, nor from changes in the basic data. But
14 must, in fact, be a result of the methodology.
15 Q. And based on your experience as a statistician,
16 would that kind of instability in the results cause
17 you to call the model that you have chosen into
18 question?
19 A. It would.
20 Q. And why is that?
21 A. If the process you're studying stays stable, you
22 have fairly large sample sizes for every year, the
23 data look very similar. One would expect the result
24 of the analysis also to be stable.
25 If they're not, then they must not be
GRUTTER -vs- BOLLINGER, ET AL
106
1 reflecting the data or the policy, something else
2 must be going on. I mean you want to know what it
3 was.
4 Q. Let me now sort of bring this discussion full circle
5 and ask you, you know, now that we've looked at your
6 views on Dr. Larntz's work, how do your simulations
7 and your results bear on your views of the results
8 that Dr. Larntz reported?
9 A. My results using the simulations, of course, are
10 asking a much less challenging question. We're
11 asking what's the causal impact of the policy on
12 those who apply, rather than asking how much are the
13 people who are doing the admissions weighing
14 different factors. So it's a more modest question.
15 And the results, however, I think are
16 very informative about the potential consequences of
17 policy changes both for those who apply and for the
18 overall diversity of the class.
19 The results are very stable over
20 years, they can be bounded with truly minimum
21 assumptions. I mean essentially the only assumption
22 that we're making is that the probability of
23 admission for majority candidates will not go down
24 if race is abolished as a factor in the admissions.
25 So, with minimal assumptions, we have
GRUTTER -vs- BOLLINGER, ET AL
107
1 stable results that I think are very informative on
2 the question of how using race affects the people
3 who apply.
4 In Professor Larntz's case, he was
5 trying to answer a much, much more difficult
6 question. Which is to use these limited statistical
7 data, to try to make inferences about how people
8 were making decisions, when the people who were
9 making the decisions have a great deal of
10 information that we just don't have access to.
11 And so his results in addition as we
12 see, because of methodological reasons, using
13 certain subsets of data, not using others, creating
14 unstable results, are somewhat problematic. But
15 that's not really the big point here.
16 The big point is that we can't really
17 answer the question he posed with the data at hand.
18 And I think that's the key, that's at least the
19 story.
20 Q. In your view, would it be fair to say based on the
21 data, that race is a predominate factor in the
22 admissions process?
23 A. No. The data do not suggest that race is a
24 predominate factor in the admissions process.
25 Q. And, in your view, what do the data show about the
GRUTTER -vs- BOLLINGER, ET AL
108
1 impact that considering race has on the admissions
2 process?
3 A. They show that the impact on minority candidates
4 would be quite substantial, we suspect. And the
5 impact on majority candidates would be very modest.
6 Q. And that's the impact of changing to an alternative
7 race blind policy?
8 A. Correct.
9 MR. DELERY: Your Honor, at this
10 point I would move Exhibits 184 through 194, the
11 charts we used, into evidence.
12 MR. PURDY: No objection.
13 THE COURT: Received.
14 MR. DELERY: And no further
15 questions.
16 THE COURT: Does the Intervenors have
17 any questions?
18 MS. MASSIE: Yes, we will. It might
19 be a good time to break for lunch though.
20 THE COURT: Okay. We'll break for
21 lunch and you'll still get an hour and 15 minutes.
22 Why don't we argue those motions before lunch, it's
23 not going to take but a couple of minutes to do that
24 and then we'll break for lunch.
25 Let the record reflect that we have
GRUTTER -vs- BOLLINGER, ET AL
109
1 Plaintiff's motion in limine to exclude certain
2 Intervenor witnesses.
3 Have the Intervenors, rather than
4 spending a lot of time arguing about those you
5 intend to call, have you made a decision as to any
6 of those that have been objected to at this point?
7 MS. MASSIE: Just one second. What
8 we know for sure is that we won't be calling all the
9 ones. We won't be calling all the ones who have
10 been object ed to.
11 For example, at most we anticipated
12 calling one of the four law professors who are
13 listed as fact witness. Those being Margaret
14 Montoya, Sumi Cho, Marjorie Schultz and Charles
15 Lawrence.
16 Frankly I don't think we're going to
17 have a chance to call anyone of those four people,
18 but if we do call one it will be one.
19 In other words, there's no chance
20 that all four of them are being called. Beyond that
21 I think it's extremely unlikely of the triad of John
22 Hope Franklin, Thomas Sugrue and Eric Foner, that we
23 would seek to call more than two of those three
24 witnesses. And we might, in fact, call only one of
25 them.
GRUTTER -vs- BOLLINGER, ET AL
110
1 THE COURT: Okay. I just thought we
2 were wasting time on that.
3 MR. KOLBO: Counsel. Your Honor,
4 Kirk Kolbo again for the Plaintiff. Our concern is
5 if I can just be brief about this.
6 In conversations with counsel for the
7 University, I think I've learned that they're
8 probably going to finish with their case next Monday
9 or so. So we'll spend about five trial days
10 altogether between the Plaintiff's case and the
11 University's case.
12 And Ms. Massie had listed, I
13 understand now, it's being diminished, but I think
14 some 27 witnesses. We're concerned about that for a
15 number of reasons.
16 And I'm not here today to talk about
17 cumulative testimony, I think that might be
18 appropriate at some point.
19 But even given the fact the court has
20 given each side 30 hours just seems to be at some
21 point testimony, I think, in any particular subject
22 can become cumulative. But that again is not really
23 what my concern today is.
24 For the witnesses that I have
25 mentioned here, it seems to me that they are, as far
GRUTTER -vs- BOLLINGER, ET AL
111
1 as I can tell, they are addressed subjects that are
2 clearly outside the scope of the Court's order in
3 the trial of the case.
4 And, in fact, I think I've been
5 somewhat conservative in this. I think I can
6 actually find some of their other witnesses who
7 appears to me, at least, can only be relevant on
8 matters that are outside the scope of the trial
9 here, and I tried to focus on a few that I thought
10 made this point best.
11 For the most point, the witnesses
12 here that we have mentioned--and I'm not going to go
13 through them one by one, your Honor, unless you
14 would like me to.
15 THE COURT: No, you don't have to. I
16 have read everything.
17 MR. KOLBO: They seem to fall into
18 two categories. One is witnesses who will testify,
19 they're all academics, I think, in one fashion or
20 another.
21 And I'm not trying to at this point
22 exclude any of the Intervenors, we don't think that
23 their testimony given the scope of the trial is any
24 more relevant then Ms. Grutter's is at this stage,
25 we're not objecting to their testifying in court.
GRUTTER -vs- BOLLINGER, ET AL
112
1 I am concerned about the academics
2 that seem to be offered on matters that are outside
3 the scope of the trial in two areas in particular.
4 One, it appears that the Intervenors
5 plan to have experts testify very generally about
6 historical race relations in this country, history
7 of discrimination.
8 Those aren't matters in dispute, your
9 Honor, we don't dispute Plaintiffs in this case
10 there's a long history of discrimination against
11 minorities in this country.
12 We think given the Supreme Court's
13 precedence those kinds of important issues simply
14 can't rise to a compelling governmental interest to
15 justify racial preferences.
16 And for that reason, that kind of
17 testimony isn't needed or relevant here. And these
18 experts really, even though they're talking about
19 discrimination generally, they're not experts, as I
20 understand it, your Honor, that are offered as
21 experts on standardized testing or cultural bias
22 with respect to grades and test scores.
23 They're much more general then that.
24 And it seems to me we just don't need to spend time
25 with that.
GRUTTER -vs- BOLLINGER, ET AL
113
1 The other general category, your
2 Honor, and I'm actually more concerned about the
3 second category than the first, because it seems to
4 me this open up all kinds of possibilities, as far
5 as where this trial might head.
6 Ms. Massie has identified a number of
7 experts who would be really experts on the question
8 of whether diversity has educational value.
9 And we all know that that issue has
10 been taken under advisement by the court as a matter
11 of law. But a number of these witnesses, as far as
12 I could tell, could only be relevant on that
13 subject.
14 And a number of these professors, for
15 example, a larger majority of the group we may only
16 hear one from. But it just seems to me that that's
17 not the issue that's before the court. We have not
18 prepared ourselves to try it at that level at this
19 point.
20 And the other thing that concerns me
21 on that, your Honor is, I did see the University
22 file and I guess our response as well to our motion
23 with respect to the Intervenors.
24 And they indicated that if
25 Ms. Massie, if the Intervenors get to put on
GRUTTER -vs- BOLLINGER, ET AL
114
1 evidence of the educational value of diversity, well
2 then they ought to be able to too. And it seems to
3 me that we're on a whole different ball game at that
4 point.
5 THE COURT: That's Professor Franklin
6 and Montoya and so forth?
7 MR. KOLBO: Yes, I think a number of
8 them tend to cross into that area. All of the last
9 four Montoya, Sumi Cho, Marjorie Schultz,
10 understanding that only one may be called now but
11 here is what each are supposed to testify about.
12 Why it's necessary to have a critical
13 mass of minority students for those students who
14 achieve their full potential.
15 That's just isn't one of the issues
16 as I understand it that we're trying in that narrow
17 scope of the trial. Those are our concerns, your
18 Honor.
19 THE COURT: Thanks.
20 MS. MASSIE: Thank you, Judge. Let
21 me say first that we don't intend to have any of
22 these witnesses if they're called to testify about
23 the educational benefits of diversity.
24 THE COURT: Good, because I was going
25 to rule in the Plaintiff's favor, because it's not
GRUTTER -vs- BOLLINGER, ET AL
115
1 an issue here.
2 MS. MASSIE: We know that.
3 THE COURT: And the University take
4 exception to it also, because they feel that they
5 have those put some witnesses on, that they have
6 strong witnesses in which they have not put them on
7 and they were not limited. Go on.
8 MS. MASSIE: That's understandable
9 and we have no dispute with any of that. All of
10 these witnesses will go to questions that you've
11 identified at the trial.
12 They'll go to why there's a score gap
13 in the LSAT. For example, they're go to why you
14 have to take the kind of breaks in admissions to
15 move toward fairness and equality in law school
16 education. That will go to the extent to which you
17 have to take in account of race in admissions.
18 In that regard, they're not
19 completely unlike Syverud on the question of extent.
20 Mr. Kolbo was just objecting to the idea that
21 critical mass is still an issue in this case.
22 But you the other day ruled that
23 Kent Syverud could testify based in part on his
24 testimony on critical mass, which has to do with the
25 extent to which race has to be taken into account in
GRUTTER -vs- BOLLINGER, ET AL
116
1 the law school admissions process.
2 THE COURT: Well, I've limited it to
3 a very narrow--well, I didn't limit it, but they
4 intended to call him for the very limited and narrow
5 issues. But, go on.
6 Well, I'll tell you how I'm going to
7 rule, because it's not a secret. I am going to
8 again indicate, and I just indicated to you, that as
9 to those issues that are not relevant here shouldn't
10 be presented.
11 I'm not going to tell you exactly how
12 to present your case and I don't intend to do that.
13 You have 30 hours total, I don't know how many you
14 have used so far.
15 However, I will accept any during the
16 testimony before you spend the money bringing these
17 folks in, remember that I'm ongoing to allow that
18 which is really relevant.
19 And again history, the history of
20 discrimination in this country the Plaintiffs are
21 not disputing the effects of that and so forth.
22 So, those issues that are before me,
23 and again I don't know everything that these
24 witnesses are going to testify to. I think I have a
25 total of maybe 20 pages here between all parties in
GRUTTER -vs- BOLLINGER, ET AL
117
1 relation to this particular motion.
2 So I'm going to deny the motion.
3 However, with the understanding that both Plaintiff
4 as well as the University may make objections as to
5 relevance. If it's not relevant, I'm going to take
6 a pretty hard line on that.
7 MS. MASSIE: That makes sense.
8 THE COURT: Okay. We'll be back at
9 1:15.
10 (A brief recess was taken.)
11 - - -
12
13
14
15
16
17
18
19
20
21
22
23
24
25
GRUTTER -vs- BOLLINGER, ET AL
118
1 (Court back in session.)
2 THE COURT: Okay, you ready?
3 MS. MASSIE: Yes, your Honor.
4
5 CROSS-EXAMINATION
6 BY MS. MASSIE:
7 Q. Hi, Dr. Raudenbush.
8 A. Hi.
9 Q. I never introduced myself earlier, we've never met
10 before today, is that right?
11 A. That's right.
12 Q. I'm going to ask you, Dr. Raudenbush to turn to your
13 original report in this matter which is date
14 January 22, 1999, and I believe it is at 145, do you
15 still have it in front of you?
16 A. No, I don't actually.
17 MS. MASSIE: Is it okay if I
18 approach?
19 THE COURT: Please.
20 BY MS. MASSIE:
21 Q. And this is the same report you were looking at
22 earlier today?
23 A. Yes, this is my first report.
24 Q. Got you. Could you turn for me to page seven,
25 please.
GRUTTER -vs- BOLLINGER, ET AL
119
1 A. Okay.
2 Q. What I would like to do here is if--
3 MS. MASSIE: Judge, do you have page
4 seven in front of you?
5 THE COURT: I sure do.
6 MS. MASSIE: I'm not going to bother
7 about projecting it then.
8 THE COURT: I have it.
9 BY MS. MASSIE:
10 Q. I'm going to ask you to read the paragraph that
11 begins with the word nor, close to the bottom of
12 that page if you would. And I'll just ask you to
13 read the full paragraph, and then I'm going to ask
14 you to explain a little bit what you mean by it?
15 A. Okay. "Nor does the report--
16 Q. (Interposing) Can I stop you right there, I
17 apologize. When you say the report, you mean?
18 A. The report of the--the first report, I believe it
19 was, of Professor Larntz.
20 Q. Thanks.
21 A. "Nor does the report consider the possibility that a
22 given value on the index has a different meaning on
23 average for different ethnic groups.
24 A candidates score on the LSAT and on
25 GPA may be viewed as reflecting motivation, aptitude
GRUTTER -vs- BOLLINGER, ET AL
120
1 and prior educational opportunities.
2 Presumably, if one person has had
3 more opportunities than another and if both have the
4 same index score, the second person must have a
5 higher level of aptitude plus motivation.
6 Admissions officers have some
7 information on each component and can use that
8 information to ensure that those accepted for
9 admission are uniformly capable with regard to
10 motivation and aptitude, but diverse not only with
11 respect to ethnicity, but also with respect to prior
12 educational opportunity.
13 A sensical statistical analysis of
14 the admissions process should use all of the
15 available data to explore whether and by what means
16 the University has been able to achieve such goals.
17 But statistical analysis that equate
18 test scores in prior cases with aptitude and
19 motivation to learn law, would overstate the
20 predictor of validity of LSAT in particular.
21 That model will also be predicated on
22 the assumption that prior educational opportunity
23 had no role to play, or that the access of minority
24 applicants to prior educational opportunities is, on
25 average, equal to the prior educational
GRUTTER -vs- BOLLINGER, ET AL
121
1 opportunities of Caucasian."
2 Q. Dr. Raudenbush, what do you mean by aptitude, do you
3 mean in borne intellectual capacity of some kind?
4 A. Not necessarily. It's hard to nail down the things
5 that cause people to have different aptitudes for a
6 particular subject or area of study.
7 Q. So you're not necessarily referring to something in
8 a person from birth?
9 A. Not necessarily.
10 Q. And do you still agree with the views that you
11 expressed in this paragraph?
12 A. Yes.
13 MS. MASSIE: That's all I have.
14 Thanks.
15
16 CROSS-EXAMINATION
17 BY MR. KOLBO:
18 Q. Good afternoon, Dr. Raudenbush.
19 A. Good afternoon.
20 Q. We met once, I think, before telephonically doing
21 your deposition in the midst of a blizzard, I think?
22 A. That's right. It was the longest phone call I ever
23 had.
24 Q. Just for the record, my name is Kirk Kolbo and I
25 represent the Plaintiff. One of the lawyers
GRUTTER -vs- BOLLINGER, ET AL
122
1 representing the Plaintiff in this particular
2 lawsuit.
3 Am I correct that you have, first of
4 all, you've been, I think, very careful several
5 times to indicate that you have not done any
6 statistical analysis for the purposes of quantifying
7 the extent to which race is used in the admissions
8 process at Michigan Law School, correct?
9 A. That's correct.
10 Q. But you have conducted a statistical analysis to
11 assess whether race is an important factor in the
12 Michigan Law School process, correct?
13 A. An important predictor in terms of just the
14 correlation, not in terms of a factor. It depends
15 on what you mean by factor. But statistically to
16 predict.
17 Q. I think you used the word earlier today, causal
18 factor?
19 A. We look at the causal not of race per se, but of
20 using race. Of a policy that uses race relative to
21 another policy that doesn't. We look at the causal
22 effect of those two policies.
23 Q. And you look to determine whether that was the
24 important causal effect in this case, correct? The
25 use of race, that is?
GRUTTER -vs- BOLLINGER, ET AL
123
1 A. The use of race, yes.
2 Q. And you have concluded, have you not as a general
3 matter, that race is important causal effect with
4 respect to admissions decisions that are made at the
5 Michigan Law School, correct?
6 A. It's not quite that simple, if I may explain. That
7 the magnitude of the affect is quite large for
8 minority applicants on average, but not for majority
9 applicants.
10 Q. But you have concluded, have you not, that there
11 would be very important consequence for the racial
12 composition of the Michigan Law School if race were
13 not a factor in the admissions process, correct?
14 A. That's correct.
15 Q. And your testimony has been there would be a larger
16 impact for the group of minority students, relative
17 to the group of non-minority students, correct?
18 A. That's correct.
19 Q. And you, in fact, have attempted to quantify the
20 extent to which that is true, correct?
21 A. I have.
22 Q. And you have concluded that there would be, as I
23 understand it, very dramatic consequences in terms
24 of the reduction of minority students at Michigan
25 Law School if we went from the current policy to a
GRUTTER -vs- BOLLINGER, ET AL
124
1 hypothetical race neutral policy, correct?
2 A. Correct.
3 Q. You have, I think, used terms like substantial and
4 sharp in terms of introduction?
5 A. Yes.
6 Q. Would you agree that that would--I think Ms. Munzel
7 the other day, I don't know if you were here for her
8 testimony, but she suggested, she's the current
9 Admissions director, that there would be a
10 devastating drop in minority admissions if the law
11 school were to go to a race neutral system and
12 everything else in the system remained the same?
13 A. I wasn't present during that testimony.
14 Q. Would you agree with that characterization?
15 A. Well, the word devastating, I'm sure people might
16 disagree as to what would be devastating. Some
17 people might be devastated and others might not.
18 I think statistically the numbers,
19 the expected reduction in the the average
20 probability of admission which I showed, is quite
21 substantial.
22 Q. Just to use one more lay person's term. Would it be
23 fair to say that the consequences would be enormous?
24 A. Again, statistically the word enormous, it's a very
25 subjective word. I'd rather just stick with the
GRUTTER -vs- BOLLINGER, ET AL
125
1 numbers and language I used in my own reports.
2 Dramatic is about as far as I went, I don't think I
3 necessarily used the word enormous.
4 Q. Maybe dramatic but not enormous? Substantial and
5 sharp?
6 A. Substantial and sharp. I think the size of it is
7 quite clear.
8 Q. And you did as you say, I've just been using terms
9 that I understand. I'm a history major normal not a
10 statistical. I'm not sure I even took a course in
11 statistics, to tell you the truth.
12 But you didn't rely simply on English
13 language, but you quantified your findings and you
14 spent some time doing that that afternoon?
15 A. That's correct.
16 Q. And I want to ask you about some of that a little
17 bit later as well.
18 A. Okay.
19 Q. If I may use the Defendant's board over here. This
20 is Exhibit 184. This was the first slide or, I
21 guess, the first graphic that you displayed this
22 morning, Dr. Raudenbush. And I want to just ask you
23 a couple of questions about it.
24 This is a display of what, I guess,
25 would be descriptive statistics, correct?
GRUTTER -vs- BOLLINGER, ET AL
126
1 A. Yes, these is descriptive statistics.
2 Q. And one of the things that you demonstrated through
3 this is for the year 2000, for example, there were
4 about 14.4 applicants who were minority students
5 UMS. I will use the same shorthand. Of all
6 minorities, you only know Asian Americans, for
7 example, are not included.
8 Generally what we're talking about is
9 UMS or minority students in the context of your
10 testimony, correct?
11 A. That's correct. They're not classified as being
12 underrepresented minority students, because the law
13 school policy doesn't include them in the category
14 of people who, as I remember, have been historically
15 discriminated against and would likely be
16 underrepresented.
17 Q. And if I could just have the same understanding that
18 you had with Mr. Delery this morning, unless I say
19 otherwise, when I talking about minority students,
20 I'm talking about the underrepresented minority
21 student groups that we talked about earlier, okay?
22 A. Fair enough.
23 Q. And the year 2000 about 14.4 percent minority
24 students applied, and about 35 percent of those were
25 admitted, correct?
GRUTTER -vs- BOLLINGER, ET AL
127
1 A. That's right.
2 Q. And about the same percentage of the enrollments
3 following the yield 14.5 percent enrolled about the
4 same as the total number of applicant pool, correct?
5 A. That's right.
6 Q. And then you got the numbers up there for Caucasians
7 as well?
8 A. Right. Actually for non-minority.
9 Q. Non-minority, right. Which does include Asian
10 Americans, I think?
11 A. Yes. As well as those whose ethnicity us unknown.
12 Q. I just want to understand that this analysis, for
13 example, the total numbers of minority students who
14 are admitted, that's without regard to any
15 consideration of relative qualifications, correct?
16 A. That's right. These are just simply descriptions of
17 who was admitted.
18 Q. This involves no analysis that compares the
19 credentials of those two groups, minority students
20 and non-minority students?
21 A. That's correct.
22 Q. Now, would you agree that grades and test scores are
23 very important predictors for all applicants at the
24 law school?
25 A. Yes, I would.
GRUTTER -vs- BOLLINGER, ET AL
128
1 Q. Grades and test scores are very important predictors
2 for minority students?
3 A. Yes.
4 Q. Grade and test scores are very important predictors
5 for non-minority students?
6 A. Yes.
7 Q. And is it fair to say then that based on what you
8 have seen in the data, the law school certainly uses
9 grades and test scores to make decisions with
10 respect to all applicants?
11 A. Certainly test scores and grades play a very heavy
12 role for all subgroups of applicants. I don't know
13 about each individual applicant, but certainly for
14 all the ethnic groups, absolutely.
15 Q. Speaking in terms of groups though, we can certainly
16 say that the law school uses, looks at and makes
17 decisions based on grades and test scores of all
18 applicants.
19 And that's a true statement with
20 respect to minorities, and that's a true statement
21 with respect to majority students, correct?
22 A. Correct.
23 Q. Is it true though that you have also found in
24 looking at the data, that the relative importance of
25 those factors, grades and test scores, in at least
GRUTTER -vs- BOLLINGER, ET AL
129
1 the decisions that are made out of the Admissions
2 Office.
3 The relative importance is different
4 for different racial groups?
5 A. The regression co-efficients are different for
6 different groups. Which doesn't necessarily imply,
7 however, that their relative to importance in making
8 the decisions is different.
9 The regression co-efficients don't
10 necessarily reflect the process that makes the
11 decisions. It's a statistical association that
12 we're looking at here.
13 Q. Could you get Exhibit 146 for Dr. Raudenbush. It's
14 one of your reports, maybe you have it in front of
15 you.
16 A. I have it.
17 Q. It's your report dated March 3, 1999?
18 A. I have got it.
19 Q. And I'm on page five.
20 A. Okay.
21 Q. I'm on the last full paragraph. I'm just going to
22 read, I may stop to make sure that we can--correct
23 me if I'm reading things wrong, and I also want to
24 ask you if these are true statements in your report.
25 First sentence, "The evidence of
GRUTTER -vs- BOLLINGER, ET AL
130
1 these effects", and I assume the evidence means the
2 data you looked at, correct?
3 A. Now, you're starting in the middle here. Okay, I
4 see. Go ahead.
5 Q. The evidence of these effects, and when we're
6 talking about evidence here we're talking about the
7 data you looked at, right?
8 A. Right.
9 Q. "Is presented in detail a series of logistics
10 regressions in the appendix." And you attach an
11 appendix to your report here, correct?
12 A. Correct.
13 Q. And continuing on, "These analysis show that for all
14 applicants grades and test scores are extremely
15 important as predictors of mission to the law
16 school."
17 And we just agree that that's true,
18 correct?
19 A. That's correct.
20 Q. And then you go and state, "Other factors are also
21 important for all applicants, however, the relative
22 importance of each factor differs as a function of
23 underrepresented minority status.
24 Test scores, grades, Michigan
25 residents and gender play quite different roles in
GRUTTER -vs- BOLLINGER, ET AL
131
1 determining the probability of admissions for these
2 two groups."
3 Are those all true statements?
4 A. Yes. Now, these are statements of statistical
5 association, not statements of what people are
6 thinking about when they make their decisions. But,
7 yes, that's true.
8 Q. Just to quote again, "The relative importance of
9 each factor differ as a function of underrepresented
10 minority status."
11 Is that a true statement?
12 A. That's a true statement. You need to though look at
13 the sentence just above it which talks about
14 important as predictors of admission. That's all I
15 mean to say. Otherwise I agree.
16 Q. And again I don't want to get into too much
17 technical jargon, because I won't be able to
18 understand it or ask the right question.
19 But we're talking predictors. There
20 you're assuming that, first of all, grades and test
21 stores were used as predictors in your regression
22 analysis, correct?
23 A. That's correct.
24 Q. And we can fairly assume, can we not, that they're
25 predictors because the Admissions office is using
GRUTTER -vs- BOLLINGER, ET AL
132
1 grades and test scores to make admissions decisions,
2 correct?
3 A. It seems very unlikely that they would be as strong
4 predictors as they are if the Admissions people were
5 completely ignoring them.
6 And everything I know about the
7 policies says that they're supposed to be important
8 predictors. So I guess in that sense the data are
9 consistent with the policy.
10 Q. So, if they are strong predictors, grades and test
11 scores, and if we can assume that the Admissions
12 Office is using them, and if we can conclude that
13 they have a different relative importance for
14 different racial groups, can't we draw a conclusion
15 that the Admissions Office is attaching a different
16 relative importance to those factors in considering
17 minority applicants?
18 A. No, we can't. We definitely cannot draw that
19 conclusion from these data. I can explain that if
20 you like, but we can't.
21 Q. No, I just was curiously looking for an answer.
22 A. Okay.
23 Q. Now, I guess I would like to go next, jumping over a
24 number of of your exhibits. I would like to go to
25 the--maybe I should just put it up, because I'm
GRUTTER -vs- BOLLINGER, ET AL
133
1 going to be talking about it if not immediately then
2 soon, Exhibit 187.
3 And I'm not going to draw your
4 attention to this immediately, but I hope to lead up
5 to it.
6 As I understand it, you use logistic
7 regression, the same mode of analysis that
8 Dr. Larntz used. You don't have an objection to the
9 use of that mode of analysis to form some
10 comparative analysis, do you?
11 A. No, I don't.
12 Q. And as I understand it, you made a choice--you have
13 to make a choice in using logistic regression about
14 what predictor variables we're going to use?
15 A. That's correct.
16 Q. You makes some assumptions about the fact that
17 they're probably--they probably have predictor
18 value, correct?
19 A. You do, yes. You start by hypothesizing what things
20 might actually predict the outcome, yes.
21 Q. And in this case you choose grades and test scores?
22 A. Yes.
23 Q. And it's not clear to me, maybe you can just explain
24 this to me. I think you got one regression model
25 that just uses grades and test scores. And then
GRUTTER -vs- BOLLINGER, ET AL
134
1 another one that uses a few others like residents
2 and gender and so forth, is that correct?
3 A. That's correct.
4 Q. Which of those regression analysis are equations
5 that you used in coming up with the comparative data
6 that you got there with Exhibit 187?
7 A. We used the one that had the other factors,
8 including gender and residents.
9 Q. Now, you didn't use any factors that Dr. Larntz did
10 not use, did you?
11 A. No, I don't think we did.
12 Q. You used nothing in addition to the factors that
13 Dr. Larntz used?
14 A. That's correct.
15 Q. In constructing your regression analysis?
16 A. That's correct.
17 Q. And I think you made this clear, and I think your
18 report makes it clear. There are a lot of factors
19 other than grades and test scores that go into
20 admissions decision making, correct?
21 A. That's correct.
22 Q. Nobody disputes that?
23 A. Nobody disputes it, but the policy seems to list
24 quite a large number of things.
25 Q. And you didn't take account any of those other
GRUTTER -vs- BOLLINGER, ET AL
135
1 factors other than what you did, correct?
2 A. That's right.
3 Q. And as I understand it, you came to your conclusions
4 about what would happen to minority admissions by
5 formulating a regression equation that best explains
6 admissions decisions for majority students, correct?
7 A. That's what generated these numbers, yes, that's
8 right. We checked it with another methodology, but
9 we did do what you said, correct.
10 Q. And when I say, again, I apologize if I'm not very
11 precise, it's just because I can't be in some of
12 these with my limited facility of statistics.
13 But with respect to the regression
14 analysis for majority students, what you were trying
15 to do there is to come up with an equation with
16 grades, test scores and these few other factors as
17 predictors, that best explains admissions decisions
18 for majority students?
19 A. That's correct. Actually we did estimate those for
20 both majority and minority, but in generating these
21 we actually used the resulting majority equation.
22 Q. And the equation--to get the equation that best
23 explains the result, what you're doing, are you not,
24 you're kind of working backwards to try and figure
25 out how much weight in the equation one would have
GRUTTER -vs- BOLLINGER, ET AL
136
1 to give factors, the predictor variables like grades
2 or test scores, in order to get the best explanation
3 for the result, correct?
4 A. That's true. But I'm not actually creating those
5 weights. Those weights are being estimated from the
6 data. I just want to make sure.
7 Q. Absolutely, I understand. We're working backwards,
8 you're trying to explain, you see the outcome and
9 now you're trying to work backwards with an equation
10 to understand how that outcome can be best
11 explained?
12 A. Can be best predicted.
13 Q. It's not a perfect prediction, because there are
14 other factors?
15 A. That's correct.
16 Q. But you're trying to come up with an equation that
17 best predicts what happens, correct?
18 A. That's correct.
19 Q. And to do that, or ultimately looking to get the
20 result, you end up with weights being assigned to
21 whatever the predictor values are?
22 A. That's correct. Those are estimated from the data
23 and then we use those to predict the probabilities
24 of admission, correct.
25 Q. So, there's a weight that you end up with through
GRUTTER -vs- BOLLINGER, ET AL
137
1 this equation, following based on whatever effects
2 however, that would be assigned to grades and test
3 scores?
4 A. Correct.
5 Q. As well as the other few objective factors that you
6 looked at?
7 A. That's right.
8 Q. And that was just for majority students though,
9 right?
10 A. We did it for both, but as I said the one we
11 actually used for these are listed in the equation,
12 correct.
13 Q. So, there's a separate regression equation that
14 would best explain, best predict I guess is the
15 word, admission outcome for the minority students
16 considering the same predictor variables, correct?
17 A. That's correct.
18 Q. And it's different because, again, you found that
19 there is a different relative importance, at least,
20 in terms of the effects with respect to how grades
21 and test scores are considered for these two racial
22 groups, correct?
23 A. That's correct.
24 Q. Could I ask Wayne to put up on the board, I guess at
25 this point, I think it's the second to the last page
GRUTTER -vs- BOLLINGER, ET AL
138
1 of Dr. Raudenbush's report. It's Appendix A1.
2 A. Same report we're looking at here?
3 Q. This is the same report and it's Table A1, it's the
4 second to the last page.
5 A. I have got it.
6 Q. Believe me, I'm not going to ask a lot about this
7 because I'll be totally lost very soon.
8 But do I understand here you have
9 reported the equations for the two different racial
10 categories that you designed, or I should say
11 figures out the regression equation for?
12 A. That's correct.
13 Q. On the left-hand column on the left side there is
14 the regression equation for whites and Asians?
15 A. That's correct. Yes, that actually does include
16 Native Americans. I checked on that, it include
17 actually white--I'm sorry, whites and Asians and
18 blacks, Hispanics and Native Americans, yes.
19 Q. Whites and Asian America?
20 A. That's correct.
21 Q. That's the non-minority group?
22 A. Right.
23 Q. That's the the equation on the left?
24 A. Yes.
25 Q. That you used to assess your--to form your
GRUTTER -vs- BOLLINGER, ET AL
139
1 conclusion about what would happen under a race
2 neutral system, correct?
3 A. That's correct.
4 Q. And on the right is a separate equation, correct?
5 A. Correct.
6 Q. A different equation. And that's the equation that
7 best predicts admissions decisions for the minority
8 students, at least, considering these predictor
9 variables?
10 A. Correct.
11 Q. And they're different again because we have
12 concluded in looking at the data that the relative
13 importance of these factors is different, correct?
14 A. Yes. The intent of which they actually predict the
15 outcome is different. Of course you can see that
16 they are, as we said before, very important for both
17 groups.
18 Q. Right.
19 A. But they're different.
20 Q. Very important and very different, correct?
21 A. They're different.
22 Q. Okay.
23 A. The extent of the difference, of course, is somewhat
24 different in different years. But there tend to be
25 some difference.
GRUTTER -vs- BOLLINGER, ET AL
140
1 Q. One can measure the extent of the difference,
2 correct?
3 A. In all of my reports, in fact, I give them year by
4 year as you can see.
5 Q. Right. You have, I think, you just said it, but you
6 did very similar analysis for each of the admissions
7 data, years that we have, correct?
8 A. That's right.
9 Q. Your supplemental report dated March whatever one
10 you're looking at, March 3, 1999, this is for these
11 particular years. Later reports, I think, include
12 this data at least for '99 and 2000, I think,
13 correct?
14 A. That's right.
15 Q. Did you do these--I don't even remember, did you
16 construct these separate regression equations for
17 all of the years in question?
18 A. I did.
19 Q. Okay. In all cases they were different equations?
20 A. They were sometimes more similar and sometimes they
21 generally were statistical different. They were
22 different more often then they were the same or
23 similar.
24 Q. Am I correct that one of the premises in your
25 opinions, is that you use the term in your report
GRUTTER -vs- BOLLINGER, ET AL
141
1 and I'm looking at page seven, if you want to take a
2 look at this.
3 Underrepresented minority students
4 are disadvantaged with respect to GPA, test scores
5 and other factors relative to other racial groups,
6 that's language that you use there?
7 A. Yes, that's language that I used, right.
8 Q. When you used the word disadvantage there, is that a
9 statistical term?
10 A. Yes.
11 Q. And does that mean that they have lower test scores
12 and grades?
13 A. They have lower means on something that's related to
14 the outcome, correct. In this case grades and test
15 scores.
16 Q. And it was because--and you have got a reference to
17 other factors here. What other factors did you
18 determine that minority students were disadvantaged
19 with respect to relative to other racial groups?
20 A. Actually in this data set I actually don't, in the
21 law school data set, I can't think of any other
22 factors where I have evidence that they were
23 disadvantaged with possible exceptions of alumni
24 status.
25 This was perhaps a typo, because I
GRUTTER -vs- BOLLINGER, ET AL
142
1 wrote a similar report in the undergraduate case
2 where there were other facts.
3 Q. I understand that, that's happened to me a lot.
4 A. It's embarrassing, but that seems to be what's here.
5 Q. And do I understand that there was a significance
6 for your comment on that in your report, that there
7 was a disadvantage with respect to these factors.
8 The significance of that was that you
9 concluded that to eliminate minority status as a
10 consideration while maintaining these other criteria
11 in place, I assume to be test scores and grades,
12 will presumably reduce the probability of admission
13 of minority students possibility substantially?
14 A. That's correct.
15 Q. So, that was, at least, one of the reason you
16 thought that might occur?
17 A. That's right.
18 Q. And you, in fact, did an analysis that confirmed
19 your conclusion in that regard, is that correct?
20 A. That's correct.
21 Q. Correct. I think that's all I'm going to need for
22 that. You may want to put up again the graphic I
23 had up, Exhibit 187.
24 Now, that we have got some foundation
25 for it, I've had a chance to ask you some questions
GRUTTER -vs- BOLLINGER, ET AL
143
1 about the equation, the non-minority equation that
2 you used to assemble your differences and
3 probability. I want to ask you some questions about
4 the conclusions.
5 You have concluded for the year 1995
6 that if one were to take the--if one were to run the
7 minority admissions, the data that you got to the
8 majority regression equation, one would see a drop
9 in minority student admissions from .26 to .04, is
10 that correct?
11 A. In '95.
12 Q. In '95?
13 A. Right.
14 Q. That's about, what, about a 85 percent drop?
15 A. It's something like that, I guess.
16 Q. Very substantial?
17 A. Yes, very substantial. It's actually the biggest
18 one in all the years.
19 Q. And that's just as a result of changing one factor
20 in the admissions process, correct?
21 A. That's correct.
22 Q. That's just under the assumption that you remove
23 race as a factor in admissions?
24 A. Correct.
25 Q. And the other part of the assumption, or at least
GRUTTER -vs- BOLLINGER, ET AL
144
1 one other assumption is, that everything else stays
2 the same in the admissions process, correct?
3 A. Well, we have to assume that everything else stays
4 the same. What we're able to actually control for
5 unfortunately is simply grades and test scores. And
6 Michigan residents and gender which are not very
7 important as predictors.
8 So, essentially we're using grades
9 and test scores, and we're forced to assume that the
10 other things are operating in the same way and not
11 correlated with those.
12 Q. As we talked about earlier, there's a lot of factors
13 that are considered?
14 A. Right.
15 Q. Grades and test scores are important for everybody?
16 A. Right.
17 Q. They're very important. And your assumption would
18 by running the minority applicants through the
19 majority equation, that the importance to the LSAT,
20 for example, would stay the same, correct, as it is
21 today?
22 A. That's right. The predictive power of it would be
23 the same, correct. Not the importance of the
24 admissions decision, but the statistical importance
25 in doing the predictions would stay the same.
GRUTTER -vs- BOLLINGER, ET AL
145
1 Q. It would remain as important a factor under your
2 comparative analysis as it is today, at least, for
3 the majority students, correct?
4 A. Correct.
5 Q. And other factors--that's true with all the other
6 factors as well, grades would have the same?
7 A. Grade will, yes. The other factor we is data on,
8 yes.
9 Q. Everything is held constant?
10 A. Right.
11 Q. With only one exception?
12 A. Right.
13 Q. And that's the removal of the consideration of race
14 in the process?
15 A. Right.
16 Q. And just with that one factor you get this very
17 substantial drop in the admissions, correct?
18 A. Correct.
19 Q. Now, you've indicated that there would be a change
20 also with respect to the non-minority students, and
21 you've indicated that whereas with minority students
22 there would be a negative impact.
23 With respect to non-minority
24 students, there would be a positive impact with
25 respect to more offers of admission?
GRUTTER -vs- BOLLINGER, ET AL
146
1 A. The average probability of admission would go up
2 with majority students if race were eliminated as a
3 factor.
4 Q. And you've made clear that the difference in the
5 average probabilities would be substantial, that is
6 a much greater impact with respect to the minority
7 groups versus the majority?
8 A. Yes, their probabilities would go down quiet
9 substantially. The majority probabilities would go
10 up, but not very much.
11 Q. But I want to be clear about something else. That's
12 comparing the two groups, correct?
13 A. Correct.
14 Q. Now, with respect to individuals, the change is
15 going to be--there's going to be a number of
16 individuals who are minority students who are not
17 going to be admitted because of this change to a
18 race neutral system, correct?
19 A. Correct.
20 Q. And do I understand your analysis to be, or the
21 consequence of your analysis to be, that those seats
22 in the class will then be filled by non-minority
23 students?
24 A. What actually would happen would be that there would
25 be a small addition to the number of seats that
GRUTTER -vs- BOLLINGER, ET AL
147
1 about 3000 people would be competing for.
2 Q. But the absolute numbers would be on a one to one
3 ratio, wouldn't they?
4 A. The absolute numbers in terms of the composition of
5 the final student body?
6 Q. For every minority?
7 A. For those admitted, yes. The number admitted, under
8 our simulation the numbers of minority
9 students--this is an assumption, that the numbers of
10 minority students--the difference of the number
11 admitted would be equal to the gain in the number of
12 majority students admitted. I think that's your
13 point.
14 Q. The absolute numbers are on one to one ratio,
15 correct?
16 A. Correct.
17 Q. So, for every minority student who is out, there's
18 presumably a majority students who wins?
19 A. Somewhere out there somebody will win that extra
20 seat that about 3000 people competing for.
21 Q. We just don't know who that is?
22 A. That's correct.
23 Q. Just like we don't know which minority student is
24 going projected?
25 A. Right. We assume that the credentials of those
GRUTTER -vs- BOLLINGER, ET AL
148
1 people would be evaluated, and the people with the
2 best credentials would be the ones who win the seat.
3 Q. Can we just, maybe, take an example. I don't
4 suppose you have a calculator with you?
5 A. I do.
6 Q. You may not need it for this one.
7 A. I hope I don't, but I have one.
8 Q. I think the first slide we had up there with the
9 total numbers, yes, let's take that one. And I'll
10 keep in front of me, I think it would be hard to
11 have two of them up there at the same time.
12 But in 1995 we had 262--actually,
13 let's go down to the bottom.
14 A. These are all 2000 data.
15 Q. I'm sorry, 2000. Let's go to the bottom of the
16 chart you have 484 minority applicants, correct?
17 A. Correct.
18 Q. And what I see, at least, on my Exhibit 187 is that
19 under the current system, 35 percent were offered
20 admission?
21 A. Right.
22 Q. And under, at least, in Exhibit 187 if you go to
23 Policy B the race neutral system we go to four
24 percent admitted.
25 A. Okay. No, that wasn't in 2000. 2000 it was ten
GRUTTER -vs- BOLLINGER, ET AL
149
1 percent.
2 Q. I'm sorry, I'm getting confused. Ten percent.
3 A. Okay.
4 Q. Ten percent under the race neutral system in the
5 year 2000?
6 A. Right.
7 Q. And so that's actually 35 percent is actually two
8 and a half times ten percent, right?
9 A. Three and a half, I think.
10 Q. I told you I would embarrass myself what that math.
11 We have even got to the statistical.
12 Can you calculate what that would
13 mean in the year 2000 in terms of the number of
14 minority students?
15 A. If the proportion admitted were ten percent out of
16 the 484 who applied, there would be 48 admissions.
17 Q. Okay.
18 A. 48.4. But we can't admit that point four person, so
19 we make it 48.
20 Q. I understand. And that's about 120 fewer
21 admissions?
22 A. Right.
23 Q. And so presumably there would be 120 more offers of
24 admissions to other racial groups?
25 A. Correct.
GRUTTER -vs- BOLLINGER, ET AL
150
1 Q. Did you ever assess what the overall effect on the
2 minority composition, and here I'm going to use the
3 term differently. I'm going to use the term, let's
4 include Asian Americans and any other groups other
5 than UMS students.
6 A. So majority you're referring to?
7 Q. Well, Asian Americans and I presume there are other
8 racial groups that are not necessarily majority
9 groups. Caucasians, Asian Americans?
10 A. Yes.
11 Q. Did you do any analysis--
12 A. (Interposing) Maybe we should take Asians so I can
13 understand where you're going.
14 Q. Okay. Let me back up a little bit. I'm wondering
15 whether you did any analysis to determine what the
16 overall impact would be on minority admissions at
17 the law school, as a result of going to a race
18 neutral system, including Asian American in the
19 definition of minority students?
20 A. No, I didn't do that. I just used the definition of
21 underrepresented minority students that was in the
22 1992 policy.
23 Q. Is it fair to presume that because Asian Americans
24 are sort of considered as majority students for
25 these purposes here, that some of those seats that
GRUTTER -vs- BOLLINGER, ET AL
151
1 would open up, would be seats which would be
2 competed for Asian Americans?
3 A. I think I now understand the analysis that you might
4 be suggesting. I did do an analysis, I didn't
5 record it in my reports, but I did do an analysis of
6 how the average probability of admission would
7 change for Asian Americans under Policy A and
8 Policy B. And the increase is exactly the same as
9 it is for Caucasians.
10 Q. Actually I was just trying to figure out what the
11 total minority population would be of the law school
12 under the race neutral system that you have
13 suggested as a hypothesis?
14 A. No, I didn't use any definitions of minority other
15 than the underrepresented minority status definition
16 that appeared in the 1992 policy.
17 Q. Did you consider using any other frame of reference
18 to assess what would happen under a race neutral
19 system, other than the current system?
20 A. No. You know, our Policy A was always based on the
21 data that we had. And Policy B was the simulated
22 alternative.
23 Q. Would it have been possible to have assigned
24 different values to LSAT and grade points as
25 predictors, and determine what the different effect
GRUTTER -vs- BOLLINGER, ET AL
152
1 would be for different values?
2 A. It would be very possible to do that.
3 Q. You didn't do that though?
4 A. I didn't do that.
5 Q. And one could, I suppose even, eliminate the LSAT
6 and then make some predictions based on what would
7 happen to minority enrollment, correct?
8 A. Well, the problem with that would be, if the current
9 policy used both LSAT and GPA and then we came up
10 with an alternative policy, how would we know what
11 the association would be between GPA and the
12 probability of admission.
13 See we always base that on the data
14 that were available, and the data that was available
15 were based on the current policy that uses both LSAT
16 and GPA.
17 Certainly you could say, let's just
18 assigned a weight of 1.0 to the grade point average
19 generate predicted values and then do the
20 simulation, and then compare that to what we do now.
21 Q. Well. I understand that point. In other words,
22 it's kind of a purely academic exercise to
23 understand where you might want to be with respect
24 to different values, correct, different weights?
25 A. Well, generally what I would try to do as simulation
GRUTTER -vs- BOLLINGER, ET AL
153
1 would be to make sure that if I'm comparing two
2 policies, Policy A and B, would be make sure that
3 I'm simulating important policy relevant
4 alternatives.
5 And that's how I would construct the
6 A and B in this case. We tried to tailor it just in
7 such a way that it would really be relevant to the
8 policy options that are kind of at issue.
9 Q. Well, one possibility is if the Admissions Office
10 decided to use LSAT scores and grades in a less
11 important way then they are today, and they were
12 admissions decisions generated as a result of that
13 process, one could then construct a new regression
14 equation that best predict outcomes under that new
15 system, correct?
16 A. Yes. In general we ought to be able to try to, if
17 we had a realistic policy alternative, try to
18 simulate what would happen as long as we have the
19 data that are relevant to that alternative. Some
20 data that are, at least, relevant for that
21 alternative policy.
22 Q. You made, in your testimony you made some statements
23 or assumptions about what might happen to the yield
24 under the hypothetical system you proposed?
25 A. Yes. I speculated that changing the average
GRUTTER -vs- BOLLINGER, ET AL
154
1 probability admission for a particular group could
2 effect the yield. Although, to highlight really the
3 point in our simulations, we assume that the yield
4 would be the same. Which may well not be true.
5 Q. That's something that's kind of outside your
6 expertise, isn't it, like how the yield might be
7 effected?
8 A. Not entirely outside of my expertise. I know that
9 in graduate admissions that I'm involved in that
10 people who have the highest grades and test scores
11 generally have the lowest probability of accepting
12 an offer.
13 And we assume, and it seems to be
14 true when they check into it, that they have offers
15 from other highly prestigious universities.
16 Q. I have just a couple of questions for you. Well,
17 let me back up.
18 You've indicated, I think, you've
19 acknowledged that there are problems with these
20 models regression equations, because there are all
21 of these other factors that are out there, and one
22 can't take account of all of them?
23 A. That's correct.
24 Q. But my understanding is, you're pretty confident
25 about your conclusions here with respect to what
GRUTTER -vs- BOLLINGER, ET AL
155
1 would be the consequences of going to a race neutral
2 system, am I correct in that assumption?
3 A. We're able to check, to create what we call bounds,
4 upper and lower bounds, on the causal effect for the
5 majority students, that don't depend at all on the
6 assumptions of the regression equation.
7 And in that sense to sort of bound
8 how much uncertainty get through into the system by
9 the fact that we don't know all of this stuff.
10 Q. And as a result, I think you said you did another
11 check on this as well, for example?
12 A. Yes.
13 Q. And as a result of these checks that you get, you're
14 pretty confident then about your results here, in
15 terms of what would be the relative probability
16 changes in going to a race neutral system?
17 A. Yes. And I might add just to reiterate something I
18 did mention this morning. We're more confident
19 about the bounds for majority students under the two
20 policies then we are minorities students, because
21 the bounds are narrower.
22 Q. Are you confident that there will be a--all other
23 things being held equal, and if one were to go to
24 the race neutral system under the hypothesis that
25 you have explained, are you pretty confident that
GRUTTER -vs- BOLLINGER, ET AL
156
1 there would be these very dramatic substantial sharp
2 reductions in the admission of minority students?
3 A. Yes, I'm quite confident that they would be
4 substantial. The bound is just--there's a little
5 more uncertainty there because the bound is wider,
6 but, yes,
7 Q. And just to be clear on this. This decline that you
8 have testified to it, the only thing that accounts
9 for that, at least, in your statistical analysis is
10 the removal of race as a factor in the admissions
11 process, correct?
12 A. Yes. Well, the question of why is it that there
13 would be a substantial reduction is more complex
14 then what you just said. But the policy change
15 that's generating it is this change, that's right.
16 I mean again, it's contingent
17 upon--basically there's two other factors that are
18 critical in making that, in effect, large.
19 One is the fraction of all people who
20 are admitted. The fact that this is a selective law
21 school, lots of people apply, most people are
22 rejected.
23 And secondly, the fact that there is
24 a strong association between grids and test scores
25 on the admissions decision. And if those two things
GRUTTER -vs- BOLLINGER, ET AL
157
1 are true, then differences between two groups
2 minority and majority in those grids and test scores
3 can translate into--even if those differences
4 aren't very large, can translate into big
5 differences in the probability of admission under
6 the new policy.
7 So, I just wanted to create a little
8 context there to understand why this is occurring
9 when you change the policies, because of other
10 conditions in the system.
11 Namely the selectivity of it and the
12 reliance on grades and test scores that make that
13 happen.
14 Q. And the relevant importance of those factors for
15 race is relevant as well, correct?
16 A. Not for race, it's just that minority and majority
17 students have different means on two variables that
18 are very strongly predictive of admissions.
19 If those variables weren't so
20 strongly predicted of admissions, then you wouldn't
21 see such a big difference.
22 Also if the school were less
23 selective, if the number of people admitted were
24 more similar to the number who apply, you wouldn't
25 see those differences.
GRUTTER -vs- BOLLINGER, ET AL
158
1 So, to understand those differences,
2 you really have to take into account the dynamics of
3 the system. So you change one variable and it has
4 that effect because of how the system works.
5 Q. Just a couple of questions on odds ratios. Do you
6 use odds ratios in the statistical work that you do?
7 A. I do.
8 Q. You sometimes find odds ratios report calculated
9 value of infinity?
10 A. I don't.
11 Q. You've never seen one of those?
12 A. Typically to have an odds ratio that has an infinity
13 you have to have a very small sample size. You have
14 to have one or the other of the two groups. You
15 have either all have the event occur, or none of
16 them have the event occur.
17 And then the data I worked at, I have
18 never found a data set where--I mean I don't usually
19 analyze data where, let's say, everyone drops out of
20 high school, or everyone goes to college, those
21 kinds of data. I've never analyzed data that have
22 those kinds of numbers. So I wouldn't see those
23 large odds ratios in those kind of data.
24 Q. Well, just let me ask you. If you were to see that,
25 let's say, just forget about law school for a
GRUTTER -vs- BOLLINGER, ET AL
159
1 minute. If you were to see a series of odds ratios
2 analysis, perhaps we've all used a drug, for
3 example.
4 Which ten patients were administered
5 a drug and ten of them were cured. And in another
6 hospital a placebo was administered to 50 people and
7 one of them was cured.
8 Would one be able to compute relative
9 odds for those two groups?
10 A. Yes, you could.
11 Q. Wouldn't relative odds be infinity?
12 A. Generally if we had those data we wouldn't do a
13 statistical analysis, because if everybody is cured
14 we don't need statistics to know it.
15 Q. Well, as a matter of statistical principals, do
16 those figures yield comparative information?
17 A. Those figures?
18 Q. Yes.
19 A. Sure. If I had a drug that everybody was cured and
20 then basically nobody was cured, that would be
21 statistical information, right.
22 Q. It yields comparative statistical information?
23 A. Yes.
24 Q. If it could calculate the value of the odds ratio,
25 of the relative odds infinity?
GRUTTER -vs- BOLLINGER, ET AL
160
1 A. Yes.
2 MR. KOLBO: May I confer with my
3 colleague, your Honor?
4 THE COURT: Of course.
5 MR. KOLBO: Your Honor, I have no
6 further questions.
7 THE COURT: Defense has any other
8 questions?
9 MR. DELERY: Just a couple of
10 questions, your Honor.
11 THE COURT: Sure.
12
13 REDIRECT EXAMINATION
14 BY MR. DELERY:
15 Q. Professor Raudenbush, Mr. Kolbo used the term weight
16 several times when talking about the co-efficients
17 in your regression equations?
18 A. That's correct.
19 Q. Do you recall that? Am I right that weight has a
20 technical term?
21 A. It does.
22 Q. I mean a technical meaning in that sense?
23 A. It does.
24 Q. Do those co-efficients correspond to the weight that
25 the Admission officers give the various factors when
GRUTTER -vs- BOLLINGER, ET AL
161
1 they're making admissions decisions?
2 A. No.
3 Q. And when you use the term relative importance as a
4 factor in the discussion earlier, am I right that
5 you were not talking about the relative importance
6 that Admissions officers gave the factors when they
7 were making admissions decision?
8 A. You're correct, I was not doing that.
9 Q. We saw a moment ago the two regression equations
10 that Mr. Kolbo discussed with you?
11 A. Yes.
12 Q. Can you conclude anything from the fact that there
13 are two equations about how the factors are actually
14 being considered by the Admissions officers when
15 they're making the decisions?
16 A. No, you can't.
17 MR. DELERY: No further questions,
18 your Honor.
19 THE COURT: Okay, you may step down.
20 (Witness excused.)
21 THE COURT: Thank you. I forgot, who
22 is your next witness?
23 MR. PAYTON: My next witness is
24 Dennis Shields, I think he stepped out.
25 THE COURT: No problem.
GRUTTER -vs- BOLLINGER, ET AL
162
1 MR. PAYTON: I'll go get him.
2 THE COURT: We can take a little
3 break now and go from there. Take our afternoon
4 break.
5 (Court in recess.)
6 (Court back in session.)
7 THE COURT: You maybe seated.
8 Dean Shields.
9 DENNIS SHIELDS,
10 was called as a witness at approximately 2:40 p.m.,
11 after having been first duly sworn to tell the
12 truth, the whole truth and nothing but the truth.
13
14 DIRECT EXAMINATION
15 BY MR. PAYTON:
16 Q. Would you state your name for the record?
17 A. Dennis J. Shields.
18 Q. Mr. Shields, where do you currently live?
19 A. I live in Durham, North Carolina.
20 Q. And what do you currently do?
21 A. I'm the assistant Dean for Admissions and Financial
22 Aide at Duke University School of Law.
23 Q. And how long have you been at Duke?
24 A. I've been at Duke three years. I moved there in
25 January of 1998.
GRUTTER -vs- BOLLINGER, ET AL
163
1 Q. It is just about the anniversary?
2 A. Yes.
3 Q. And what did you do before you were the director of
4 Admissions and Financial Aide at Duke?
5 A. I was the assistant dean and director of Admissions
6 at the University of Michigan Law School.
7 Q. And when did you come to the University of Michigan
8 Law School?
9 A. I started as of July of 1991.
10 Q. And before you were the director of Admissions at
11 the University of Michigan Law School, what did you
12 do?
13 A. I was the assistant dean for Admissions and
14 Financial Aide at the University of Iowa Law School.
15 Q. Okay. And when did you start at the University of
16 Iowa in Admissions?
17 A. I just started as a third year law student in 1981.
18 Q. You went to Iowa Law School?
19 A. Yes.
20 Q. You graduated from Iowa Law School?
21 A. Yes, I did.
22 Q. So, you started as a third year law student, when
23 did you start after law school, when was the first
24 time you started in the Admissions Office?
25 A. Right after I graduated, that May.
GRUTTER -vs- BOLLINGER, ET AL
164
1 Q. And at what point did you become in charge of
2 Admissions at Iowa?
3 A. I believe that was 1985.
4 Q. Is it fair to say that you have been in law school
5 Admissions for about 20 years?
6 A. This is my 20th year.
7 Q. A little scary, isn't it?
8 A. Yes.
9 Q. And you've been in charge of Admissions at three
10 schools, Iowa, Michigan and Duke for 15 years?
11 A. Yes.
12 Q. Okay. Are you in any professional organizations
13 that relate to law school admissions?
14 A. Well, I've had extensive affiliations over time with
15 the MBA, the Law School Admissions Council. I
16 currently serve as council member on the Council of
17 Legal Education and Opportunity. I'm a member of
18 the National Bar Association.
19 Q. The Law School Admissions Council, what is that?
20 A. Well, that's the entity that is essentially
21 responsible for the administration of the law school
22 admissions test, and the law school data assembly
23 service.
24 Q. Okay.
25 THE COURT: Is the name of that
GRUTTER -vs- BOLLINGER, ET AL
165
1 organization is the law?
2 A. Law School Admissions Council.
3 THE COURT: Thank you.
4
5 BY MR. PAYTON:
6 Q. And what has been your affiliation with it?
7 A. I served on a number of different committees,
8 Minority Affairs Committee, I was chair of the Audit
9 Committee. I was a member of the board of the
10 Law School Admissions Council for a total of six
11 years, I believe.
12 Q. I want to focus your attention on your tenure as the
13 director of Admissions at Michigan.
14 How did you come to be the director
15 of Admissions?
16 A. I believe the associate dean for Student Affairs,
17 Susan Ekland, wrote me a letter in late 1990 or
18 early 1991, and invited me to submit a resume for
19 consideration.
20 Q. They found you?
21 A. Yes.
22 Q. And you then underwent a process--you heard
23 Professor Lempert and President Bollinger discussing
24 how you came to actually be hired?
25 A. Yes.
GRUTTER -vs- BOLLINGER, ET AL
166
1 Q. You were present in court for that testimony?
2 A. Yes, I was.
3 Q. Was that accurate?
4 A. Yes.
5 MR. PAYTON: I'm not going to go over
6 that again.
7 THE COURT: Yes, that's fine.
8 BY MR. PAYTON:
9 Q. Did you know Allan Stillwagon?
10 A. Yes, we knew each other. Not well, but knew each
11 other.
12 Q. How did you know Allan Stillwagon?
13 THE COURT: I have one question
14 before you get into that. How did you happen to get
15 into admissions, just fall into it, or was it
16 like--I'm curious?
17 A. My mentor who was then the dean of Admissions at
18 Iowa, and is now the dean of the law school at
19 Ohio State asked me if I wanted, he had a half time
20 position, and asked me if I wanted to do it.
21 And I actually thought when I
22 graduated he had made it a full time job that I
23 would do it for a couple of years until I decided
24 what I wanted to be when I grow up.
25 So, now it's 20 years later area and
GRUTTER -vs- BOLLINGER, ET AL
167
1 either I haven't grown up, or I haven't decided what
2 I want to be.
3 THE COURT: Or it's something you
4 really like. Again, I am just amazed because, you
5 know, it's such an important position in the
6 acadame. But most as I have heard so far have come
7 from areas out of the acadame.
8 BY MR. PAYTON:
9 Q. How did you know Allan Stillwagon?
10 A. Well, he was the director of Admissions at the
11 University of Michigan. There is a lot of
12 recruiting travel which you do, we be at the same
13 events, the annual meeting of the law school
14 Admissions Council. So we bump into one another.
15 Q. So you knew him before you came to Michigan?
16 A. Yes.
17 Q. Did you ever have a conversation with him about how
18 he did admissions at Michigan?
19 A. No.
20 Q. Once you came to Michigan in the summer of 1991, did
21 you call him up and ask him what had been going on?
22 A. No.
23 Q. Actually have you talked to Allan Stillwagon since
24 you became the dean at Michigan in the summer of
25 1991?
GRUTTER -vs- BOLLINGER, ET AL
168
1 A. We had pleasantries here in the courtroom.
2 Q. That is on Tuesday?
3 A. On Tuesday. Other than that, I have not even laid
4 eyes on him since 1990 maybe.
5 Q. Now, one of the first things that happened when you
6 came to Michigan, was that the dean, then
7 Dean Bollinger, put you on the faculty Admissions
8 Committee that was charged with coming up with a new
9 policy, is that right?
10 A. Yes.
11 Q. I'm not going to go into how the committee
12 functioned either, you heard Professor Lempert and
13 you heard Dean Bollinger.
14 Did they accurately describe how that
15 happened and how the committee functioned?
16 A. Yes. I appreciated that.
17 Q. But I do want to ask you this which is, what the
18 opportunity to serve on that committee looked like
19 to you having just arrived at the University of
20 Michigan Law School?
21 A. Well, it was a tremendously exciting time for me. I
22 was coming to what is already believed to be one of
23 the finest law schools in the country. I had been
24 asked to take on a major role. And I think
25 President Bollinger admitted this is an important
GRUTTER -vs- BOLLINGER, ET AL
169
1 aspect of the law school life.
2 I was going to be working on this
3 committee with a very distinguished group of faculty
4 members that had expertise in areas that I knew less
5 about then they did you.
6 But it was also an opportunity for me
7 to walk into a situation where in many ways I had
8 more expertise then they had. And to help them
9 think through this very important subject for the
10 law school. It was very exciting.
11 Q. The expertise that you had was about admissions?
12 A. Absolutely.
13 Q. About law school admissions?
14 A. About law school admissions.
15 Q. And you were the person that had the expertise on
16 the committee in that area, is that right?
17 A. I don't think there's anybody else on that committee
18 that had one-tenth the experience I had actually in
19 Admissions.
20 Q. Okay. I actually don't intend to go over the policy
21 again either, I think we've had enough of that. But
22 I do want to ask you about some of your
23 contributions to what's in the policy.
24 In the policy we have heard testimony
25 and we have seen summaries about Student X, I think
GRUTTER -vs- BOLLINGER, ET AL
170
1 there's two Student Xs, a Y and a Z. And we heard
2 those were actual student files, is that correct?
3 A. Yes.
4 Q. And how did those student files come to the
5 attention of the Committee?
6 A. Well, I selected them along with a whole host of
7 others. When I arrived here I discovered that the
8 faculty had actually not read files in years, over
9 decades.
10 And as part of the process it was
11 important for them, I thought, to get an actual feel
12 for what it was like to review a file. What was in
13 it, what kind of things to consider and that kind of
14 thing.
15 So I selected a whole range of files
16 for them to peruse.
17 Q. At the very end of the policy, actually it's the
18 attachment to the policy there's a grid, you know
19 what I'm talking about?
20 A. Yes, I do.
21 Q. And there's been some testimony about that format.
22 And that format of the grid, I think, was what
23 Mr. Larntz used to create his model of cells?
24 A. Yes.
25 Q. You were present for this?
GRUTTER -vs- BOLLINGER, ET AL
171
1 A. Yes.
2 Q. Let me ask you this, the grid that's at the end of
3 the policy, was it designed with the idea of being
4 able to show how race played a role in any
5 Admissions process or decision?
6 A. No, not at all.
7 Q. I want to talk to you a little bit about how you
8 went about implementing this Admissions policy.
9 When you were figuring out how to
10 implement this policy, and how to train people in
11 your office about how they should go about their
12 jobs once we're after the adoption of the policy and
13 that's in the spring of 1992, you created a document
14 which I believe is entitled Gospel According To
15 Dennis?
16 A. Yes, that's right.
17 Q. What can I say?
18 A. Little did I know.
19 Q. Now, we have got Dennis. Could you hold that out.
20 THE COURT: Is that Exhibit 5?
21 MR. PAYTON: I believe it's
22 Exhibit 5.
23 BY MR. PAYTON:
24 Q. You recognize this document?
25 A. Yes, I do.
GRUTTER -vs- BOLLINGER, ET AL
172
1 Q. The Gospel According To Dennis?
2 A. Yes.
3 Q. Written October 3rd, 1992, what was the purpose of
4 this document?
5 A. Well, I had in that year, I anticipated in the
6 future that I would have people who had never read
7 the law school Admissions files in the past, that
8 would be involved in evaluating and providing some
9 assessment of files for me as I went about my
10 business in making decisions.
11 So, this was a document I created for
12 them, as part of their preparation for that process.
13 Q. Okay. I take it you read a lot of files yourself?
14 A. Yes, a lot.
15 Q. Is it fair to say that you read most of the files?
16 A. Most of the files.
17 Q. Okay. And who else would read files in your office?
18 A. There was always a number two person in my office,
19 the assistant or associate director of Admissions.
20 And then there were, depending on the year and the
21 staffing kinds of things, up to one or two other
22 people on my staff that read files.
23 Q. And you would give them this document?
24 A. I would give them this document, as well as a copy
25 of the Admissions Policy.
GRUTTER -vs- BOLLINGER, ET AL
173
1 Q. Okay. Is that all you give them?
2 A. Yes.
3 Q. That's it?
4 A. Well, they'd look at the bulletin and that's it.
5 Q. Did you ever tell them that we're trying to get X
6 percent of underrepresented minorities?
7 A. I don't think I've ever said that to anyone.
8 Q. Okay. That just never--nothing at all?
9 A. Absolutely not.
10 Q. You gave them these two documents. Who else got
11 these two documents if there were going to be
12 members of the Admissions Committee that would read
13 files, would they get these documents?
14 A. I thought it might be a little presumptuous for me
15 to give this kind of document to a law school
16 faculty member. I'm sure you're familiar with--I
17 think you even taught law school. So you would know
18 how they would receive a document like this.
19 If I have known it was going to be
20 used in something like this, I might have
21 appreciated that.
22 Q. Well, before I go into this then, let me just ask
23 you a few questions about how you got along with the
24 faculty in implementing the policy.
25 You served on the Committee and I
GRUTTER -vs- BOLLINGER, ET AL
174
1 take it that was some relationship with the faculty,
2 is that right?
3 A. Right. I worked very close with the faculty members
4 on the Committee. I might back up, I think when I
5 arrived at Michigan since I was new to the
6 institution, it was very important for me in my job
7 as the dean in charge of Admissions, to get to know
8 the institution well.
9 And so I made all kinds of effort to
10 interact with faculty. I sat in on faculty meetings
11 as a member of the dean's staff,
12 Q. How often did faculty meetings happen?
13 A. I couldn't give you a precise number, but probably
14 at least two-thirds of the Fridays of every term had
15 a faculty meeting.
16 Q. Okay.
17 A. And I would go to probably half of them when I was
18 in town.
19 Q. Okay.
20 A. And I would go to lunch with faculty members, I
21 would make an effort to go to social events that
22 were for faculty members so I could get to know
23 them. Several of the faculty members invited me
24 over to dinner at their homes.
25 And so I thought as an ongoing basis
GRUTTER -vs- BOLLINGER, ET AL
175
1 it was my job to stay in tune with various aspects
2 of the law school, and provide plenty opportunity
3 for them to interact with me about what I was doing.
4 Q. This is a two-way relationship?
5 A. Yes, that's the way I viewed it.
6 Q. Now, Ms. Munzel testified that you had trained her
7 in how to review files and how to do Admissions, is
8 that accurate?
9 A. Yes, that's true.
10 Q. You hired her?
11 A. I hired her.
12 Q. And she started reading files, she was your No. 2?
13 A. Right.
14 Q. So she knew this document pretty well?
15 A. Well, she was supposed to have read it.
16 Q. Now, I take it it's not going to disappoint you to
17 learn that the Gospel According To Dennis has been
18 retired?
19 A. Not at all. I would assume that somebody else would
20 put it a little different spin on it.
21 Q. The Gospel According to Dennis, it begins on this
22 first page.
23 A. Yes.
24 Q. You see--it actually starts, this is the first page
25 but it says four at the top, but this is the first
GRUTTER -vs- BOLLINGER, ET AL
176
1 page?
2 A. Right.
3 Q. So, starting on what says page four, the first page
4 of this up at the top, it talks about we are trying
5 to select, do you see that?
6 A. Is that the first paragraph?
7 Q. It says under Philosophy.
8 A. Okay.
9 Q. You see right here, "We are trying to select from
10 the specially well credential pool of candidates
11 those that show the most promise."
12 A. Yes.
13 Q. Is that why this is a tough thing to do?
14 A. Absolutely. Making decisions on candidates,
15 particularly at a school like Michigan, you have a
16 pool of candidates that are very, very strong in
17 almost every way.
18 Q. And the end of that same paragraph it says, rather,
19 do you see that, "Rather we must begin with the
20 numbers and go forward from there to scrutinize the
21 essays and letters of recommendation."
22 A. Yes.
23 Q. "As well as considering extracurricular and work
24 experience, to look for candidates that show
25 intellectual talents, leadership ability and
GRUTTER -vs- BOLLINGER, ET AL
177
1 academic acumen which augers for a lively
2 intellectual educational community and important
3 contributions to the profession."
4 Do you see that?
5 A. Yes.
6 Q. That's what you wanted everyone to be able to pick
7 out when they did their job here?
8 A. Everyone that read files, that was the purpose, the
9 mission of the endeavor.
10 Q. Go to the next page, page five. You see the first
11 full paragraph that says, given all this?
12 A. Yes.
13 Q. "I try to read each file with an open mind and try
14 to find something that distinguishes the candidate
15 and provide some reason to consider them
16 affirmatively for admission." Okay?
17 A. Yes.
18 Q. Is that how you went about doing it?
19 A. Yes. Look, I would suspect that there are people
20 who think the process is one where you always look
21 for--first, look for a reason not to admit someone.
22 I tend to want to think positively about each
23 candidate, to try to find some reason to act in
24 their favor.
25 Q. Okay. If you go down to the bottom you'll see it
GRUTTER -vs- BOLLINGER, ET AL
178
1 says the basic approach and it just list things?
2 A. Yes.
3 Q. Things that you look at, the LSAT, GPA,
4 undergraduate institution, trends in grades?
5 A. Yes.
6 Q. And then it goes on to it says comparative rank.
7 And then it says, "If the index shows a significant
8 improvement from freshman/sophomore to junior/senior
9 academic performance."
10 You see that?
11 A. Yes.
12 Q. What's index mean there?
13 A. Well, the index I think we heard a little bit about
14 it earlier.
15 Q. Was this about the index score, because you don't
16 have an index score for freshman/sophomore?
17 A. No, I'm basically talking about the trend in the
18 grades there. That if it's going upward, if it's
19 going downward, if it's sort erratic, that kind of
20 stuff.
21 Q. Let me sort of go to where you were just about to
22 go. The index score, which is some formula that
23 relates LSAT, GPA with first year grades?
24 A. Yes.
25 Q. Do you use that in actually reviewing the individual
GRUTTER -vs- BOLLINGER, ET AL
179
1 file?
2 A. No.
3 Q. Is it in the file?
4 A. No.
5 Q. Why won't you use it?
6 A. Well, there are better tools to assess that in each
7 file, that you actually have to look at the academic
8 record. You have to look at the transcript, look at
9 the law school data assembly report that gives you a
10 wealth of information about each candidate in the
11 undergraduate institution. That kind of thing.
12 So, it's really not a particularly
13 useful number to look at when you're assessing a
14 file.
15 Q. Okay. When Ms. Munzel testified, and she went over
16 a file in some detail, and actually I was going to
17 show you a file but now I'm not.
18 I just want to ask you, is that the
19 way you were reviewing files and you trained people
20 to review files what you saw her do?
21 A. Yes.
22 Q. This document the Gospel, it's written right after
23 the policy went into effect in October of 1992?
24 A. Right.
25 Q. Did you ever revise it or this one just stayed?
GRUTTER -vs- BOLLINGER, ET AL
180
1 A. That was it. I usually had other things to do, to
2 sort of look back at this.
3 Q. Okay. I have looked through the entire document in
4 some detail, and there is no mention of race in here
5 at all?
6 A. That's correct.
7 Q. Why not?
8 A. Because I think when you're reading a file, that's
9 not the primary consideration as you're making
10 judgments about it. It's the things that I talk
11 about in there.
12 Q. Okay. Now, the admissions policy that we have all
13 spent some time looking at, the 1992 policy. That's
14 a policy about all of the admissions, isn't it?
15 A. Right.
16 Q. And the Gospel is also a document about all
17 admissions, isn't it?
18 A. Absolutely.
19 Q. And so you use the Gospel and the policy to guide
20 you in making all admissions decisions, is that
21 right?
22 A. That's absolutely right.
23 Q. Was there some minimum criteria for grades and test
24 scores that you needed before you would read a file?
25 A. No. Every file deserved to be read. And so that's
GRUTTER -vs- BOLLINGER, ET AL
181
1 what we did.
2 Q. Now, we also heard some testimony, I believe from
3 Ms. Munzel, about a comment sheet that was filled
4 out at the end of a file. I think the second file
5 she looked at she had someone else's comments, you
6 remember that?
7 A. Yes.
8 Q. And she actually read off some of the information on
9 the comment sheet, so we heard what kind of
10 information was on there.
11 Did you keep comment sheets?
12 A. No.
13 Q. What happened to them?
14 A. I had them thrown away.
15 Q. When did you throw them away?
16 A. At the end of the admissions year, that was my
17 instruction. Whenever the files were going to leave
18 our direct control for the admitted students that
19 ended up matriculating, the file went down to the
20 Registrar's Office.
21 And for those students who applied
22 and either were denied or chose not to come, they
23 went to a storage area. And when they left our
24 immediate control, those comment sheets were
25 removed.
GRUTTER -vs- BOLLINGER, ET AL
182
1 Q. And thrown away?
2 A. Yes.
3 Q. Was that just a standard policy you had?
4 A. Yes.
5 Q. By the way, was it the same policy you had at Iowa?
6 A. Yes.
7 Q. Now, I think we have seen some numbers about the
8 application flow that comes through the office, it's
9 3000 to 4000, some number like that?
10 A. Yes.
11 Q. You're telling me that every year you and some small
12 number of people on your staff read all three to
13 4000 files?
14 A. Absolutely.
15 Q. And reviewed them to make judgments?
16 A. Absolutely.
17 Q. And how did you decide which factors made a
18 difference, I mean there's a whole range of things
19 that are in the policy and in your Gospel memo about
20 things you ought to look at, how did you decide?
21 A. We had to sit down and read the whole file and make
22 a judgment based on everything that you saw there.
23 There was no one thing, you had to look at the
24 transcripts, contemplate the test scores, think
25 about the undergraduate institution, read the essays
GRUTTER -vs- BOLLINGER, ET AL
183
1 that were there, read the letters of recommendation
2 and arrive at an overall conclusion about an
3 individual file.
4 Q. There's a lot of discretion that goes into this, is
5 that right?
6 A. Yes, there is.
7 Q. Is that a good thing?
8 A. I think it is a good thing. With the guidance that
9 you have from the faculty of the law school, you're
10 implementing what they want to do.
11 And it's important to look at the
12 whole person in making a judgment about whether or
13 not to admit them to law school.
14 Q. Now, how did you take race into account? You're
15 reading a file, how did you take race into account?
16 A. Well, you read the whole file. It was one of
17 several, a number of factors you might take into
18 account.
19 Just as if you might take into
20 account the trend in grades, the rigor of the
21 curriculum. There was no specific way that you took
22 it into account.
23 Q. Would it be taken into account the same way in every
24 minority, underrepresented minority applicant's
25 file?
GRUTTER -vs- BOLLINGER, ET AL
184
1 A. In every file.
2 Q. Would it be taken into account in the same way?
3 A. Not in the same way. Look, the assessment of any
4 individual file is never precisely the same way,
5 there are a lot of different things that you're
6 looking at.
7 And any one factor in there, for
8 example, an especially remarkable essay may be
9 dispositive in a particular case.
10 An exceptionally strong rigorous
11 academic record may be dispositive in any given
12 case. It might be the thing that tips the balance.
13 A particularly strong LSAT score in
14 some cases, might be the thing that tips the balance
15 in favor of a candidate.
16 So, in any given file the weight that
17 you might give to any particular aspect of it, would
18 vary from other files.
19 Q. Now, as I understand it, from time to time you would
20 have a conversation with the dean, whether that be
21 Dean Bollinger or eventually Dean Lehman?
22 A. Yes.
23 Q. About how many Michigan residents you're looking
24 for?
25 A. Yes.
GRUTTER -vs- BOLLINGER, ET AL
185
1 Q. Did you ever have a conversation with either one of
2 them about how many underrepresented minorities you
3 were looking for?
4 A. No.
5 Q. You ever have a conversation with him about what the
6 range of underrepresented minorities were that you
7 were looking for?
8 A. No. Absolutely not.
9 Q. Was the manner in which race was taken into account
10 different from the manner in which, let's say, the
11 essays or leadership ability, or any of the other
12 factors were taken into account?
13 A. Well, an individual file it may carry more weight in
14 one file, it may carry less weight into another
15 file. And that was true about anything you might
16 think about in the file. In the grandest scheme of
17 things, no, it wasn't treated any different.
18 Q. Okay. Now, another of your responsibilities as dean
19 and I think we have the impression that all you did
20 everyday was sit down and read files.
21 I take it another of your
22 responsibilities was to do all the things you have
23 to do to recruit students to file the applicants in
24 the first place, okay?
25 A. Absolutely.
GRUTTER -vs- BOLLINGER, ET AL
186
1 Q. What did you do to do that?
2 A. Well, if I can. My job is to create an entering
3 class every year. And in order to do that, you have
4 to have people apply, and you have to have good
5 people apply. And if you're interested in
6 diversity, you have to have a diverse pool of people
7 to select from.
8 And so we traveled extensively to
9 college campuses, to law school recruitment fairs.
10 I made contacts, maintained my contacts with the
11 pre-law advisors on different campuses across the
12 country and corresponded regularly with them.
13 We made it a point to visit a whole
14 range of different types of institutions. We did a
15 lot of direct mail to students that we thought were
16 competitive for admission.
17 Locally I established a Minority Law
18 Day for freshmen and sophomores on the campus of the
19 University of Michigan.
20 I regularly interacted with the
21 pre-law advisors on Michigan's campuses and the
22 various student organizations that were aimed at
23 ultimately applying to law schools in the
24 undergraduate pre-law call, that kind of things.
25 Maintained the same kind of contacts
GRUTTER -vs- BOLLINGER, ET AL
187
1 with a number of other organizations on other
2 campuses around the country.
3 Q. Now, were these things that, not all of them, but
4 were some of these things, new things that you did,
5 that as far as you know hadn't been done before to
6 recruit students?
7 A. Absolutely. I took as my charge when I arrived, to
8 reenergize and to be innovative about the kinds of
9 things that we did to attract candidates for
10 admission.
11 Q. And did you do special things to try to recruit
12 underrepresented minorities to apply to the law
13 school?
14 A. Absolutely.
15 Q. What did you do?
16 A. We would visit colleges, universities where there
17 was significant population, or where that was the
18 particular mission, so to speak. Historically black
19 colleges, universities that had significant
20 representations of Hispanics, Asian Americans.
21 Other campuses we made a point to go
22 there, to find out who the undergraduate
23 organizations were that we could work with, talk to,
24 make presentations to. That kind of thing.
25 Q. And were these new things?
GRUTTER -vs- BOLLINGER, ET AL
188
1 A. Yes.
2 Q. I'd like to show you some exhibits that were used
3 with Ms. Munzel and that I used in my opening. 181,
4 182, 183 and 184.
5 A. Okay.
6 MR. PAYTON: While we're doing this,
7 your Honor, I want to offer into evidence Exhibit 5,
8 I believe, which is the Gospel According to dennis.
9 MR. PURDY: No objection, your Honor.
10 THE COURT: Received.
11 BY MR. PAYTON:
12 Q. These are charts that show data from 1997, and they
13 show all of the applicants and all of the admitted
14 students. And then separately they show it for
15 underrepresented minorities admitted,
16 underrepresented minorities rejected. Majority
17 students admitted, and majority students rejected.
18 You can put up any one. Put up any
19 one of the charts so I can just ask him
20 These are non-admitted majority
21 applicants, do you see that, Mr. Shields?
22 A. Yes.
23 Q. There were some testimony about--actually it wasn't
24 testimony, it was a representation by me about the
25 fact that there are scores on the LSAT, non-standard
GRUTTER -vs- BOLLINGER, ET AL
189
1 scores is how I referred to them, that show as zeros
2 in the data that is reported by Law Services?
3 A. Right.
4 Q. Okay. Was I right?
5 A. Absolutely.
6 Q. Could you explain what a non-standard score on the
7 LSAT is?
8 A. Well, the law school admissions test is a
9 standardized test. That is it's supposedly everyone
10 who takes it, takes it under the same conditions, et
11 cetera, et cetera.
12 Well, in fact, there are some, not a
13 whole lot, but there are some who take it under
14 non-standard conditions.
15 Most often or probably the only way
16 that that happens, is if they have some documented
17 disability. And when that happens they get more
18 time, or they get different kind of test.
19 For example, someone who has a vision
20 problem might actually have someone read the exam to
21 them. And because their taking it under nonstandard
22 conditions, the scores reported on their LSAT
23 report.
24 But in terms of the data since it's
25 non-standard they get a zero when it comes done to
GRUTTER -vs- BOLLINGER, ET AL
190
1 accounting for statistics.
2 Q. Do you see this, for example, on this chart right
3 here on the bottom axis which is zero, if you look
4 to the right and across you see a number of, I call
5 them hits, little points, those are zeros?
6 A. Right.
7 Q. Non-standard?
8 A. Test takers.
9 Q. Test takers who show on here as zeros because in the
10 data frame they show a zero?
11 A. Right.
12 Q. And that's simply how Law Services deals with what
13 you call the non-standard test?
14 A. Right. For example, when you see an LSAT report, on
15 most of the LSAT reports taken under standard
16 conditions, you get a score and an identification of
17 the percentile positioning of that score on the
18 scale.
19 In the non-standard setting, you get
20 a score but you get no percentage. And that's why
21 it shows up as a zero.
22 Q. Could you put on top of that, I don't know what this
23 exhibit is, it's 182. Can you put the companion
24 which is the non-admitted minority on that.
25 Mr. Shields, you were the director of
GRUTTER -vs- BOLLINGER, ET AL
191
1 Admissions in 1997, is that correct?
2 A. Yes, I was.
3 Q. So this is data that relates to how you ran the
4 office?
5 A. Absolutely.
6 Q. And do you see that when you look at the majority
7 and the minority plots up there, they have some
8 substantial overlap, those are my words. But do you
9 agree that they have substantial overlaps?
10 A. Yes, absolutely.
11 Q. And these were the students who did not get in?
12 A. Right.
13 Q. Are you surprised by that?
14 A. Not at all.
15 Q. Could you put up the other two. And these are the
16 admitted students, and if you pull them apart so he
17 can see the difference underneath.
18 At the top now is the majority
19 students who were admitted, and now placed on top of
20 them right now are the minority students who were
21 admitted.
22 The same scale, you see the overlap
23 there?
24 A. Yes.
25 Q. Okay. And that picture of what the class of
GRUTTER -vs- BOLLINGER, ET AL
192
1 admitted students look like in 1997, does that
2 surprise you in anyway?
3 A. Not at all.
4 Q. Did you admit, in your opinion, really good classes
5 of students, minority and majority alike?
6 A. I'm very proud of the classes that I admitted to the
7 University of Michigan Law School.
8 Q. Mr. Shields, I want to ask you about how you look
9 back on what you accomplished at the University of
10 Michigan Law School, with respect to this policy and
11 its implementation.
12 What's your reflection on how this
13 policy and your implementation work?
14 A. I think that the policy, I'm very proud of the role
15 that I had in developing it, I'm very proud of the
16 final policy.
17 I think my implementation of it and
18 attempts to accomplish what the policy asked of me.
19 I'm very proud of all of that. I don't think anyone
20 else could have done it better.
21 MR. PAYTON: Thank you, your Honor.
22 THE COURT: Intervenors, any
23 questions?
24 MS. MASSIE: None.
25 THE COURT: Plaintiffs.
GRUTTER -vs- BOLLINGER, ET AL
193
1 MR. PURDY: Thank you, your Honor.
2 Your Honor, Larry Purdy again for the Plaintiff.
3 THE COURT: Mr. Purdy.
4
5 CROSS-EXAMINATION
6 BY MR. PURDY:
7 Q. Good afternoon, Dean Shields?
8 A. How you doing?
9 Q. Good. Let me go through first, I've got a couple of
10 questions that I want to try and get to just a
11 little bit later, but let me try and walk through it
12 if I could just briefly some of the testimony that
13 you've given to us.
14 First, with regard to Exhibit 5 your
15 Gospel According To Dennis Shields?
16 A. Yes.
17 Q. Would it be fair to say that this philosophy applies
18 to every candidate regardless of his or her race
19 ethnicity?
20 A. Yes.
21 Q. And do I assume that it was your intention to apply
22 this philosophy equally to every applicant that came
23 across your desk regardless of his or her race and
24 ethnicity?
25 A. The purpose of the document was to give guidance to
GRUTTER -vs- BOLLINGER, ET AL
194
1 people reading files for me.
2 Q. Sure. And you didn't vary the way you approached
3 any file depended upon the person's race or
4 ethnicity, would that be a fair statement?
5 A. That's a fair statement.
6 Q. You know, there was a question from Mr. Payton about
7 the index scores. And I believe you said that index
8 scores is not used in the review, do you recall
9 that?
10 A. Yes.
11 Q. I believe I wrote it down and if I'm wrong correct
12 me. But I believe you said it's not useful number,
13 do you recall that?
14 A. Yes.
15 Q. But, in fact, doesn't the policy itself talk about
16 the importance of the index and the admissions
17 process?
18 A. Well, that's a short hand term for looking at the
19 law school admissions test score, and the
20 undergraduate academic record.
21 And it's not useful because it
22 doesn't give you very complete information. For
23 example, as we evaluate an academic record, the
24 quality of the school that one attends, the rigor of
25 the undergraduate curriculum that one has pursued,
GRUTTER -vs- BOLLINGER, ET AL
195
1 is not in anyway captured in that index.
2 Q. Just out curiosity because there's been so much
3 discussion about it, I actually went through the
4 policy last night and counted the number of times
5 where the word index appears.
6 Would it surprise you that the word
7 index appears in the Admissions policy 20 times or
8 more?
9 A. Not at all. Not at all.
10 Q. And we have gone over this, Exhibit 5 it doesn't
11 reference race at all, correct?
12 A. Correct.
13 Q. But the Admissions policy does?
14 A. Yes, it does.
15 Q. And, of course, you told us this afternoon that you
16 were guided in your admissions decisions by the
17 Gospel and the policy, correct?
18 A. Right. Well, the Gospel was something that I wrote
19 for the benefit of the people in my office that
20 would be reading files and providing evaluations of
21 those files, summaries of them with the files when I
22 was ready to make a judgment on them.
23 Q. I appreciate the distinction. I think what you're
24 trying to tell us is actually you were guided in
25 your decisions by the policy?
GRUTTER -vs- BOLLINGER, ET AL
196
1 A. Yes.
2 Q. There was also a question from Mr. Payton about
3 whether or not you periodically had discussions with
4 Dean Bollinger and then subsequently Dean Lehman
5 about the percentage of residents that the school
6 may be seeking each year, you recall that?
7 A. Yes.
8 Q. And, of course, it's clear that every year you did
9 have discussions trying to figure out where you
10 wanted to be, in terms of resident matriculants
11 within the class?
12 A. Right.
13 Q. And as I recall from looking at the documents, and
14 again correct me if I'm wrong. But I believe that
15 it consistently felt a third of the class plus or
16 minus were residents?
17 A. Well, it depends on what time frame you're talking
18 about. Because most of what we've been talking
19 about was from 1995 forward.
20 Q. Let me back up. From 1992 until you left in 1998,
21 did the percentage of residents that the school
22 sought to admit roughly fall in the one-third range?
23 A. Yes, give or take five percent probably either way.
24 Q. And the policy specifically mentions the preference
25 that they want to consider for Michigan residents,
GRUTTER -vs- BOLLINGER, ET AL
197
1 does it not?
2 A. Yes.
3 Q. In fact, it uses the language honoring the special
4 claims of Michigan residents to a Michigan law
5 school education, correct?
6 A. Yes.
7 Q. And, of course, that wasn't divided by race or
8 ethnicity, it's all Michigan residents?
9 A. Right.
10 Q. And then Mr. Payton asked you whether or not you
11 recalled any discussion with either Dean Bollinger
12 or Dean Lehman about whether or not there was a
13 target range for race and you told us there wasn't,
14 correct?
15 A. Right.
16 Q. But you had--
17 A. (Interposing) I didn't have any conversations with
18 them about that.
19 Q. You didn't have any conversations. But you were
20 aware of discussions about certain percentages of
21 certain underrepresented minority groups, were you
22 not?
23 A. I'm not sure what you're referring to. You mean in
24 the creation of the policy?
25 Q. Yes, sir.
GRUTTER -vs- BOLLINGER, ET AL
198
1 A. I'm familiar with those discussions.
2 Q. All right. And, in fact, if I could ask you to turn
3 to Exhibit 34.
4 A. Okay.
5 Q. Dean Shields, I am just going to ask you, when you
6 were part of the faculty policy, I'm sorry, sure the
7 policy that you were creating was the faculty
8 Admissions policy.
9 Did you periodically get copies of
10 the drafts and make your own comments?
11 A. I don't think I ever wrote comments. I saw the
12 drafts, but I generally--you have to understand what
13 my falls are like.
14 I typically visit anywhere from 20 to
15 30 different campuses, et cetera. So the time I
16 have to scratch down notes is rather limited.
17 So, usually I would try to come to
18 the meetings having read it, and then react to what
19 I had read.
20 Q. I appreciate that. And, of course, we took your
21 deposition, what, two years ago or something like
22 that?
23 A. Yes.
24 Q. And you didn't recall having any drafts and we have
25 not found any, so I'll tell you I'm not going to
GRUTTER -vs- BOLLINGER, ET AL
199
1 spring a draft of yours. Unlike counsel who sprung
2 the Gospel of Dennis on you.
3 But if you look on page 13 of
4 Exhibit 34, and this was an initial draft of the
5 policy. And while you're getting there, you do
6 recall seeing various drafts of the policy as it was
7 underway, correct?
8 A. Yes, I do recall. I don't know if I saw this
9 specific one marked up like this.
10 Q. Sure. Let me just ask you to look at the bottom of
11 the full paragraph on page 13, and I'm just going to
12 read the last sentence real quickly. In fact, I'll
13 even paraphrase it.
14 It just notes in the past we have
15 achieved the kinds of benefits that we associate
16 with racial and ethnic diversity from classes in
17 which the proportion of African American, Hispanic
18 and Native Americans members has been between eleven
19 percent and 17 percent of total enrollees."
20 Do you recall reading that from other
21 drafts of the policy while you were serving on the
22 committee?
23 A. You know, I don't know that I recall reading it. I
24 know that we talked about numbers in that process.
25 Q. All right. And let me ask you to also turn to
GRUTTER -vs- BOLLINGER, ET AL
200
1 Exhibit 32, it should be just two exhibits in front
2 of that, if you would, please.
3 A. Yes.
4 Q. And here is a memorandum from Professor Regan. You
5 know Professor Regan, do you not?
6 A. Yes, I do.
7 Q. And, in fact, this indicates that you got a copy of
8 this particular memorandum, does it not?
9 A. Yes.
10 Q. Do you recall reading this memorandum back in this
11 time frame?
12 A. I don't have a specific recollection, but I'm sure
13 that I have no question about whether I saw it.
14 Q. And so you recall Professor Regan, at least
15 reviewing at some point Professor Regan's comments
16 about whether or not to leave numbers in or take
17 them out of the policy?
18 A. Yes.
19 Q. And you recall Professor Regan's suggestion that for
20 a variety of reasons, including candor, I incline to
21 prefer to keep the numbers in and try to explain
22 what they really signify, do you recall that?
23 A. Yes.
24 Q. So, you were, at least, aware of what the faculty's
25 views were about the percentage of underrepresented
GRUTTER -vs- BOLLINGER, ET AL
201
1 minorities, or at least the size of the class that
2 they would like to attract each year?
3 MR. PAYTON: You mean the committee?
4 MR. PURDY: The committee, I'm sorry.
5 BY MR. PURDY:
6 Q. The committee?
7 A. This was Professor Regan's view point at this point
8 in time. I wouldn't want to attribute any of his
9 position to any other member of the committee,
10 okay.
11 Q. I understand that the memo that we're looking at,
12 Exhibit 32?
13 A. That's Don Regan's, Professor Regan's take on things
14 at that point.
15 Q. He was commenting, was he not however, on the
16 percentages that we see in Exhibit 34, the eleven to
17 17 percent?
18 A. I guess, I don't know.
19 Q. Do you recall discussions in the faculty meetings
20 that you attended about this policy where the
21 numbers eleven to 17 percent was discussed?
22 A. Yes, I was a very active part of those discussions.
23 Q. Dean Shields, every year while you were at Michigan,
24 you would receive numerous applications from various
25 minorities who presented stellar academic
GRUTTER -vs- BOLLINGER, ET AL
202
1 credentials, correct?
2 A. Yes.
3 Q. These were minority applicants who graduated from
4 the same range of schools as did your white nation
5 American applicant, would that be a fair statement?
6 A. There was a significant overlap in where the
7 minority students went to school that applied, and
8 where the majority students went to school that
9 applied.
10 Q. Sure. You would have minority students who came
11 from schools from the Ivy League, and you would have
12 some that came from school that I'm sure you knew to
13 be outstanding small liberal arts schools?
14 A. Very few white candidates from historically black
15 colleges.
16 Q. Any?
17 A. None that I know of.
18 Q. Okay. And you would have those that came out of a
19 lot of public institutions, and I'm talking about
20 you have minority applicants with great credentials
21 who came from public institutions like Michigan
22 State and University of Michigan, correct?
23 A. Yes.
24 Q. Just like you would white students and African
25 Americans?
GRUTTER -vs- BOLLINGER, ET AL
203
1 A. Sure.
2 Q. Minority students who followed the same tough
3 curricula, took the same tough courses, that would
4 have impressed you?
5 A. Sure.
6 Q. And if I may, I'm going to, if I could, I'm just
7 going to put page 13 of the Policy which guided your
8 work. And if you can't read it, it's Exhibit 4.
9 Q. I'm not trying to embarrass you so suggest that you
10 can't see over there.
11 A. Well, I tell you, you know, I'd hate to admit it,
12 there's probably a day I could see it.
13 Q. It's page 13, and if I could just have you turn to
14 that. And certainly you would agree, would you not,
15 Dean Shields, that there were people who were
16 members of underrepresented minority groups who you
17 would admit every year without reference, without
18 reference to their minority status, correct?
19 A. There were some, yes.
20 Q. And, of course, we know that you told told us yield
21 is a very tough problem with all applicants,
22 particularly in the upper grid cells?
23 A. I would characterize that any applicant that's
24 particularly remarkable presents a challenge in
25 convincing them to come to Michigan as opposed to
GRUTTER -vs- BOLLINGER, ET AL
204
1 other very fine law schools.
2 Q. That's interesting. Why did you have that
3 particular problem getting them to come to Michigan,
4 as opposed to other schools where they would
5 typically also be accepted?
6 THE COURT: The weather.
7 A. Well, that was some of the problems.
8 BY MR. PURDY:
9 Q. What were some of the problems Michigan faces in
10 recruiting the same kids that get accepted, to say,
11 Harvard or Yale or Chicago or UCLA or Berkley?
12 A. Those are all good options. And for a whole--well,
13 precisely for the reason I heard people think very
14 carefully about where they attend law school.
15 The size of the law school, where
16 it's located, what their long term career ambitions
17 might be. Each candidate makes sort of independent
18 choices about how that matches up with where they
19 want to go to school.
20 Q. Well, just so there's no misunderstanding. Every
21 year there--while you were here, we'll just confine
22 it to the five years, was it about five year?
23 A. Six and a half years.
24 Q. Six and a half years, I'm sorry. Every year while
25 you were serving as the dean of Admissions in
GRUTTER -vs- BOLLINGER, ET AL
205
1 Michigan, you had underrepresented minority
2 applicants who you admitted who required no
3 consideration of their race in order to obtain that
4 admission, correct?
5 A. Yes.
6 Q. And, indeed, I assume it's a fair statement that if
7 you could achieve the law school goal of enrolling a
8 critical mass of these students from this group that
9 didn't need any consideration of race, you'd happily
10 do that, would you not?
11 A. I would admit anybody who I thought was a remarkable
12 candidate.
13 Q. Sure. And if you could achieve a critical mass of
14 specifically underrepresented minority students
15 without referring to race, that would be a wonderful
16 achievement too, correct?
17 A. I hope that day comes.
18 Q. We do, we all do. That's actually the ultimate goal
19 of the policy, is it not, I mean that's what they
20 talk about on page 13. That hopefully that will be
21 exhausted at some point?
22 A. There's a lot of other goals talked about in the
23 policy, and I think that it's important to keep in
24 mind that the Policy governs overall admissions.
25 And the task that we were to undertake in that year
GRUTTER -vs- BOLLINGER, ET AL
206
1 was to rethink all of the admissions.
2 But one of the goals would be
3 to--well, I don't think it would ever be a goal with
4 this policy to not have a diverse class, okay.
5 If we ever get to the point where we
6 can achieve that without any consideration of race,
7 I think this country would be a happier place.
8 Q. But one of the goals in the policy and I won't pour
9 through the blowups and try and find it, but you
10 recall it no doubt. I think it's a previous page
11 twelve.
12 One of the goals is to enroll a
13 critical mass of underrepresented minorities?
14 A. Critical mass is part of the goal, sure.
15 Q. Sure. And in order to achieve that critical mass of
16 minority students the practice was and the policy
17 called for, a willingness to admit minority students
18 from generally lower academic qualifications then
19 majority students, isn't that a fair statement?
20 A. I think that's a fair statement.
21 Q. Do you have Exhibit 15, and if you don't have the
22 book we'll get it for you. Actually, you know,
23 before I get to that and I apologize, but you've got
24 the book.
25 Could you look at page ten, I'm
GRUTTER -vs- BOLLINGER, ET AL
207
1 sorry, Exhibit 10 first, it's a daily report.
2 A. Yes.
3 Q. Just so it's clear, you would get these periodically
4 during the admissions cycle, would you not, and this
5 would help you determine where you were in terms of
6 these offers that had been made, and where you sat
7 in terms of admissions offers that had been accepted
8 in terms of the possible things of that nature?
9 A. Yes. Early in the season I might look at this, this
10 is something I could just punch a button on my
11 computer and it will crank this out in about 15
12 minutes.
13 And in December I might get it once,
14 in January I might get it two or three times. And
15 by the time you get to April and May when the
16 deposits are rolling in, I may want to see it daily.
17 Q. I was going to say, I think I recall in your
18 deposition you talked about that it's certainly your
19 use of these types of reports increase from, let's
20 say, early March until the end of May that your
21 class was really starting to come together?
22 A. Right.
23 Q. And these reports were broken down by race so that
24 you could tell where you sat in terms of the
25 admissions from each end?
GRUTTER -vs- BOLLINGER, ET AL
208
1 A. Well, they're broken down a number of different
2 ways.
3 Q. Sure.
4 A. By race, gender, residency, non-residency, et
5 cetera.
6 Q. My only point is you did have the ability and, in
7 fact, took advantage of the ability to see how the
8 class was shaping up as you went along?
9 A. Yes.
10 Q. And part of that was to see how the class was
11 shaping up in terms of its racial and ethnic maybe
12 up, correct?
13 A. Yes.
14 Q. Let me now ask you to turn to Exhibit 15, if I
15 could, please, sir, that's where I was directing you
16 initially.
17 A. Okay.
18 Q. You know actually let me ask you, I apologize, let
19 me for a moment.
20 Do you have a view as you sit here
21 today what percent of underrepresented minority
22 students would constitute a critical mass?
23 A. No, not really. You mean you're looking for a
24 particular number or percentage?
25 Q. Even a rough percentage?
GRUTTER -vs- BOLLINGER, ET AL
209
1 A. Not really.
2 Q. Would five percent underrepresented minority
3 students constitute a critical mass in your view, as
4 an admissions expert?
5 A. I don't think so, I don't know though. I mean part
6 of that is not just my sort of assessment, the
7 assessment of other people in the law school where
8 I'm working. That kind of thing.
9 Q. And I don't mean to--would ten percent constitute a
10 critical mass?
11 A. It might, I don't know. It could.
12 Q. Looking at Exhibit 15, and I believe this is in
13 evidence.
14 THE COURT: I think it is.
15 MR. PURDY: If it's not, we'll offer
16 it.
17 THE COURT: I suspect it's in.
18 BY MR. PURDY:
19 Q. This is a copy of a grid showing all of the
20 applicants, of course, all of the LSAT and grade
21 point ranges, do you recall seeing documents like
22 this while you served at Michigan as the dean?
23 A. Yes.
24 Q. And you would use these reports each year kind of as
25 a--to compare how your current class was shaping up
GRUTTER -vs- BOLLINGER, ET AL
210
1 in comparison to your last year class, is that the
2 way you would use it?
3 A. No.
4 Q. How would you use this document?
5 A. I would look at this just to see what had happened
6 in the proceeding year. Probably--well this one--
7 Q. (Interposing) This is 1995. This was for the class
8 that entered in the fall of 1995?
9 A. The fall and summer of '95. I would typically look
10 at this kind of stuff after that class had been put
11 to bed, so to speak. Early in the fall or later.
12 And that would probably be about the only time I
13 would look at this information.
14 Q. And, in fact, this is a class that you selected, so
15 we're looking at decisions you made, correct?
16 A. Yes.
17 Q. If I could ask you to turn to the third page of this
18 document, it's the grid that shows the--in fact,
19 I'll just read it to. It's page three of
20 Exhibit 15, it's the University of Michigan Law
21 School Admissions Office, admissions grid of LSAT
22 and GPA for African Americans.
23 And I'm going to direct you down to
24 the line that begins with grade point 3.2. And
25 we're just picking it because you've been through it
GRUTTER -vs- BOLLINGER, ET AL
211
1 before, so I thought it was the easiest.
2 3.25 through 2.49 and we'll start
3 over under the LSAT score of 151 to 153, and you
4 understood that to be about the 50th percentile?
5 A. Around that, I'm not sure exactly where it was for
6 that year. But about that.
7 Q. All right. And it shows that in terms of just the
8 African Americans and we're going to start with that
9 and we're going to move up the scale on LSAT keeping
10 the grade point constant.
11 But you had seven applicants and
12 three were admitted, correct?
13 A. That's correct.
14 Q. The next, moving up to 154 to 155, five applicants
15 four were admitted, correct?
16 A. Yes.
17 Q. And then moving on to the 156 to 158. Ten
18 applicants, ten admitted, correct?
19 A. Yes.
20 Q. 159 to 160, three applicants, three admitted,
21 correct?
22 A. Yes.
23 Q. And 161 to 163, four applicants four admitted,
24 correct?
25 A. Yes.
GRUTTER -vs- BOLLINGER, ET AL
212
1 Q. And, in fact, if we go all the way across the scale
2 goes up that all the applicants obviously were
3 admitted. Let's turn to the next page,
4 Dean Shields, if we could.
5 And I want to direct you to the same
6 line, and we're going to start with the same LSAT
7 and GPA grid position.
8 MR. PURDY: And for the record, your
9 Honor, this is page four. This is the admissions
10 grid for Caucasian Americans.
11 BY MR. PURDY:
12 Q. And you will see that under the 151 to 153 where we
13 had seven African Americans, three admitted. You
14 have 24 Caucasian who applied, and zero admits,
15 correct?
16 A. Yes.
17 Q. And moving to the next column where we had
18 previously five African Americans applicants and
19 four admits, we have 21 Caucasian applicants and
20 again zero admits, correct?
21 A. Yes.
22 Q. Going up to the 156 to 158 where previously you had
23 ten African Americans applicants, all ten admitted.
24 Here there was 51 Caucasian Americans who applied
25 and one was admitted, correct?
GRUTTER -vs- BOLLINGER, ET AL
213
1 A. Yes.
2 Q. And the next column where there were three out of
3 three African Americans accepted, there was 61
4 Caucasians who applied and one was admitted,
5 correct?
6 A. That's correct.
7 Q. And going over to the next column, 126 Caucasian
8 applicants, five admitted, do you see that?
9 A. Yes.
10 Q. Dean Shields, would it be fair to assume, is it
11 accurate to assume, I'm not asking you about any of
12 the final decisions you made within these grids, but
13 the average, the difference that we see in terms of
14 the decision making with respect to African
15 Americans in these cells and Caucasians, can
16 generally be explained by the extent to which race
17 is taken in account in the admissions process, would
18 that be a fair statement?
19 A. I'm not willing to go all the way there with you
20 without reviewing the files or having the files to
21 look at. Because I couldn't be certain without
22 seeing those files again. But, at least, some of it
23 could be attributed to that.
24 Q. Let me just ask you, do you have your deposition
25 handy in front of you?
GRUTTER -vs- BOLLINGER, ET AL
214
1 A. Yes.
2 Q. If I could ask you just to turn to page 154, just
3 for a moment?
4 A. That's in the thicker one?
5 Q. It's the thick one, yes, sir. I'm going to direct
6 your attention to line 15 and I'm just going to read
7 two questions and two answers that were given to
8 you.
9 This was back on December 7, 1998
10 when my partner Mr. Kolbo who is sitting back there
11 took your deposition.
12 A. Yes.
13 Q. And let me preface and I apologize. You had just
14 gone through the same analysis with the grids as we
15 just went through?
16 A. Sure.
17 Q. Okay.
18 "Q Would it be fair to assume, is it accurate
19 to assume and I'm not asking you about any
20 individual's files here, but the average
21 here, the difference here in terms
22 of decision making with respect to African
23 Americans in these cells and Caucasians,
24 can generally be explained by the extent
25 to which race is taken into account in the
GRUTTER -vs- BOLLINGER, ET AL
215
1 admissions process?
2 A. Generally, yes.
3 Q. There might be something else in a
4 particular applicant's file, but on a
5 whole that is the explanation?
6 A. Generally that's probably true."
7 Do you recall being asked those
8 questions and giving those answers?
9 A. Well, they're here.
10 Q. But I mean those were your answers to those
11 questions, correct?
12 A. Sure.
13 MR. PURDY: Your Honor, I have
14 nothing further.
15 THE COURT: Mr. Payton.
16 MR. PURDY: Thank you, very much.
17
18 REDIRECT-EXAMINATION
19 BY MR. PAYTON:
20 Q. Mr. Shields, Mr. Purdy asked you about drafts and
21 memoranda about drafts of the 1992 policy, in which
22 there was a reference to eleven to 17 percent. And
23 you said you remembered discussions about that.
24 Do you remember your position in
25 those discussions about that?
GRUTTER -vs- BOLLINGER, ET AL
216
1 A. Absolutely.
2 Q. What was it?
3 A. I thought it was entirely inappropriate for there to
4 be numbers included because--and I said this during
5 the deliberations.
6 If, in fact, we had a pool of
7 candidates where we could not admit any specific
8 number, then that's just the way it would be.
9 And that my job was to assure that we
10 had a stronger pool of candidates, in part my job
11 was to have a stronger pool of candidates available,
12 and that we should not constrain ourselves. It was
13 fine to have an aspiration, but we should not
14 constrain ourselves to that by that.
15 So, that if there were 50 percent
16 minority in the class, that should not be looked at
17 as some sort of violation of the policy. Nor, if it
18 was less than that, it should be considered some
19 violation of the policy.
20 That, in fact, what we were trying to
21 do is make individual decisions about individual
22 candidates.
23 Q. Now, when you read a file, when you read a file when
24 you were at the University of Michigan Law School
25 and you're looking through a file, I understand
GRUTTER -vs- BOLLINGER, ET AL
217
1 there's no document that says a number.
2 But in your mind as you're going
3 through the file, do you have in your head a number
4 that you're trying to hit with respect to
5 underrepresented minorities?
6 A. Absolutely not. Absolutely not. As I read a file,
7 I'm making an independent judgment about that
8 candidate. And I may look back at the gross numbers
9 at some point in time and think, well, we're doing
10 pretty good here or we're not doing so well here to
11 whatever.
12 But as you make a decision about
13 individual files, you're not keeping in mind any
14 sort of specific target.
15 Q. Okay. Now, with respect to every single student you
16 admitted at the University of Michigan Law School,
17 and with respect to the overall classes that you
18 admitted at the University of Michigan Law School,
19 do you believe today that they were a very well
20 qualified group of students individual by
21 individual?
22 A. Absolutely. Remarkable classes.
23 MR. PAYTON: Thank you, your Honor.
24 MR. PURDY: Just briefly, your Honor.
25 THE COURT: Okay.
GRUTTER -vs- BOLLINGER, ET AL
218
1 RECROSS-EXAMINATION
2 BY MR. PURDY:
3 Q. Just very briefly to follow-up on what Mr. Payton
4 said.
5 It was expressly set forth in the
6 policy, was it not, that you no matter, what you
7 would offer admission to no applicant who you didn't
8 believe could succeed and complete the law school
9 curriculum without serious academic problems,
10 correct?
11 A. Right.
12 Q. And so if for whatever reason your applicant pool
13 didn't present you with enough residents, for
14 example, who you believe base on your review of the
15 whole file could complete the course without serious
16 academic problems, you weren't going to admit those
17 kids, correct?
18 A. Right.
19 Q. Okay. So, constrained by that, obviously you
20 wouldn't be admitting kids, you wouldn't bring
21 applicants in to the school who you didn't believe
22 could complete the program, correct?
23 A. Right.
24 Q. All right. At anytime, Dean Shields, during the six
25 and a half years that you were there, did the
GRUTTER -vs- BOLLINGER, ET AL
219
1 underrepresented minority enrollment ever drop below
2 eleven percent?
3 A. I'm not absolutely certain, but I don't think so.
4 MR. PURDY: That's all I got your
5 Honor.
6 THE COURT: Thank you, Dean.
7 (Witness excused.)
8 THE COURT: Who's your next witness?
9 MR. PAYTON: This is all I have for
10 today, as I said. My next witness Monday looks like
11 this. We're calling Kent Syverud and we're going to
12 call Dean Lehman. Those our last two witnesses.
13 THE COURT: Great. We'll recess
14 until Monday morning at nine. We'll see you Monday
15 morning at nine o'clock.
16 (Court adjourned at 3:50 p.m.)
17
18
19
20
21
22
23
24
25
GRUTTER -vs- BOLLINGER, ET AL
|
|