## The SAT is Changing — Should the LSAT be Next?

The SAT is changing. As a former SAT tutor, I like the new changes. The new SAT will stop punishing students for “guessing,” it’ll make the essay optional, and it will stop pretending like knowing all those ancient English words actually matters.

Should the LSAT change as well? Definitely.

Why The LSAT Is Good As Is

As it is, the LSAT ain’t too shabby.

First, a down-and-dirty statistics lesson. Correlation is given as a value from -1 to 1. The farther you get from 0, in either direction, the stronger the correlation. A positive value means that as one variable goes up, so does the other. Or, as one variable goes down, the other does the same.

The correlation between an applicant’s LSAT score and their first-year law school grades ranges from .12 to .56, depending on the law school. If you combine an applicant’s LSAT score with their undergraduate GPA, this correlation ranges from .3 to .62. When it comes to social science, this is a pretty strong result. For comparisons sake, the correlation coefficient for smoking and lung cancer is about .72.

So, the LSAT isn’t perfect, but it’s pretty damn good. Could it be better? I think so.

How The LSAT Should Change I: Standardize Section Order

Everyone should get the same sections in the same order, for any single LSAT administration. If you get the toughest section last, when you’re tired and your nerves are fried, you’re probably not going to do as well as if you had it first or second. For years the LSAC didn’t give people their experimental section late in the LSAT for fear of fatigue messing with their results. I think the same rational should be used in favor of standardizing section order. The proctors are already watching everyone closely for cheating. The little bit of added security you get from mixing up the sections isn’t worth sacrificing standardization.

How The LSAT Should Change II: Replace The Games With More Reading Comp

The LSAT doesn’t need a straight formal logic section. Formal logic is important, but it should be couched in better approximations of real world problems – like it is in the Logical Reasoning sections. The LSAC would do well to replace the Logic Games section with a second Reading Comprehension section. The latter is closer to the kind of work you’ll be doing every day in law school. Such a change would probably increase the LSAT’s predictive powers.

How The LSAT Should Change III: Train Better Proctors

For a test that’s as important as the LSAT (on average, 60% of your admissions decision depends on your LSAT score) there are too many awful proctors.

Just last year we had an LSAT proctor who called time 5 minutes early. We had the proctor that didn’t let anyone use their analog wristwatches. We’ve had LSAT proctors write up folks who innocently touched their pencils during a break without noting any of the context (e.g. test booklet being closed). I could go on, but you’re probably angry enough already.

The LSAC should spend more time and money training their proctors.

How The LSAT Should Change IV: Drop The Writing Sample

After a 4-hour fear and caffeine-fueled ordeal, the LSAC makes you give a sample of your writing. Some admissions folks say they never read the LSAT writing samples. Why should they? There’s a perfectly good writing sample in the applicant’s personal statement. Moreover, the LSAT Writing Sample section bares very little resemblance to anything you’ll ever have to do again, including your law school exams. More importantly, the LSAT writing samples have a meager resemblance to an applicant’s average writing, and so should not play a role in law school admissions.

How The LSAT Should Change V: Experiment More Radically

The LSAC should experiment with radical changes, like those suggest above. These changes make sense to me, but it would be great to see some real data. I’d volunteer to sit for such experimental LSATs and I’m sure plenty of others would, too.

These are my personal recommendations. If you have any of your own, don’t be afraid to post them in the comments.

## 2 Responses

1. Aaron Cohn says:

Hey Yuko,

When I saw your comments about logic games, I thought to myself, “I bet LSAC has done a study on that.”

And sure enough, here it is—the item type validity study:

http://www.lsac.org/lsacresources/research/all/tr/tr-98-01

I’ve only read the summary, but the results are interesting. As I think you suspect, games have the lowest correlation to first year law school grades of the three types of sections. Logical reasoning is the best predictor of the three; no surprises there.

But here’s where it gets interesting. Even though reading comp is a better predictor on its own than games are, games add more to the overall predictive validity of the LSAT. Reading comp scores correlate closely to logical reasoning scores, so they don’t add as much new info. Games are less strongly correlated with LR, so they add more to the predictive validity of the test as a whole.

• Yuko Sin says:

Awesome pointer Aaron. That’s really interesting.