08 January 2005

The Dangers of Statistical Methods

Vik Amar has posted an interesting (and, by its own terms) partial analysis of Rick Sander's provocative thesis that race-based law-school admissions actually harm certain racial minorities. I have a few random remarks to make from my personal observation; I don't pretend that these are statistically valid, let alone point much of a way out of the morass. (The right way out of the morass is to look at actual circumstances instead of surface characteristics; but that's for a different forum.)

  • At least in my first-year class at my law school, the observation that "the median black law student's GPA at the end of the first year of law school places him at the 7th or 8th percentile of his class" did not hold. Given the secrecy of grading and the tendency of individuals to lie about their grades, nobody can really make much of an assertion here with much confidence—except that, knowing the individuals involved and the care taken in first-year classes to grade blindly, the medians for just about any identifiable group, with one exception, were within a standard deviation of 50%. That one exception: students over 30 years old at entry to law school (unusual where we were, outside of a metropolitan area without a night program). Our median was at least the 70th percentile. I guess we weren't a protected class, then…
  • The entire study is based upon so-called "numeric indicators" of student quality. I think this an unlikely means to recognize lawyer quality, or lawyer success, until controlled for the source of the data; and that creates just a positive feedback loop. For example, the school I attended as an undergraduate had no "D" grade (A,B,C,NCr)—and very little in the way of grade inflation. A 3.3 average put one in the top quarter or so of the class. Admittedly, that's ancient history; but comparing someone with a 3.3 cumulative GPA from that school, where even the majority of the recruited athletes were National Merit Scholars, to even "better" state schools seems a bit like declaring that a Division III running back who gained 700 yards in a season is better than a Division I-A running back who gained 600. And the less said about "standardized tests" the better.
  • Finally, one of the major concerns that I have is the definition of "success" that seems to be underlying the study. It all seems to boil down to money and gravitation toward maximum-earning jobs. That is certainly one measure of success; but it is far from the only one. Those of us who have had a real life before the law—older students, and those whose backgrounds forced more responsibility upon them at earlier ages (which I suspect is disproportionately skewed toward racial minorities)—tend to look beyond the salary earned to other measures of success. An individual who was a year ahead of me, for example, has to be considered a "success in the law;" Congressman Jackson (D-IL) has, so far as I know, never set foot in a law firm, but has a lot of influence on and in the law. That's not too shabby for someone in his thirties.

I would be much happier if admissions policies would be completely race-blind and consider only the whole person for each slot. The administrative burden of doing so (and of defending particular decisions in court) is probably unrealistic. I'd be much happier with some numeric "family history" factor that considers family educational background and finances for the last couple of generations as the definition of "disadvantaged" than anything explicitly based on race. If the point is to bring more disadvantaged students into law schools (or wherever else), then it seems to me that white kids from Appalachia are as valuable to that goal as African-American kids from Barrington, Illinois.

Or, perhaps, part of the problem is the whole "reputational" issue that we can't seem to get a handle on, particularly in the law. That's the only reason I can think of that so many law professors went to Harvard or Yale; they can't all have been at the top of their classes. Or maybe it's just that the financial aid people at [three unnamed schools of very high and roughly equal reputations] were so nasty in the early 1990s… and thereby redirected a generation of students elsewhere? It worked for me!