Will the LSAT switch to an adaptive model like the GMAT or GRE?

BPP aaron-lsat-blog-cat

Earlier this week, we covered the news that LSAC is testing out the administration of a digital LSAT. Today, let’s wildly speculate about what this means for the future of the LSAT. Specifically, could the LSAT of the future be a computer-adaptive test like the GMAT and GRE? LSAC has definitely studied the possibility.

Right now, the LSAT is one size fits all. Everyone who takes the test at one particular administration gets the same four scored sections, though the order of those sections varies, as does the content of the experimental section. LSAC could, in theory, offer multiple versions of the LSAT on a given test day, but that wouldn’t change much. As soon as your test booklet gets handed to you, the test you’re going to take that day is fully determined.

This means that the LSAT is longer than it needs to be. From the perspective of the test writers, the point of the LSAT is to figure out how well the test taker can employ the underlying skills. The test needs to have different levels of questions to account for test takers’ different levels of ability. The hardest questions are needed to distinguish the very highest scorers from the merely very good. But those really hard questions don’t tell you very much about lower scorers, who mostly just get them wrong. Conversely, the easy questions are needed to separate average test takers from weaker test takers, but don’t help distinguish between high scorers—the high scorers mostly just get them right.

What if high scorers could skip the easy questions, and low scorers could skip the pointless hard ones they’re likely to get wrong anyway? That’s the promise of a computer-adaptive test, or CAT. Rather than give everyone the same questions, a CAT, uh, adapts. If you get things right, it gives you harder questions to figure out just how good you are. If you have a rough start, the test gives you easier questions to find out what you actually can do. The advantage is that the test can be more accurate, shorter, or both.

This is how the GMAT works. The questions you get within each section aren’t predetermined. The disadvantage to the test taker is that you can’t skip around. Each question demands an answer before you get to see the next one. Once you’ve made your final answer, there’s no going back. But again, the advantage is that the test is shorter than it would otherwise need to be for the same level of precision.

The GRE is adaptive in a kinder, gentler, lesser way. The GRE gives each test taker two sections each of verbal and math. The first section of each is predetermined, and you’re allowed to skip around within a section. But your performance on the first section helps determine what questions you get on the second section. Think of it as a compromise, I guess, between a traditional test like the LSAT and a truer CAT like the GMAT.

If you’re a Blueprint student, your homework is adaptive, too. In this case the point isn’t to give you a score but rather to give you questions that challenge you without going over your head.

The sample digital LSAT that’s been announced doesn’t sound as if it’s adaptive — it has the same five-section format as a standard LSAT, and LSAC hasn’t said anything about an adaptive test. But I suspect that what they’re really testing out is the digital administration method. They’re using tablets connected to a central server, and that could work for a computer-adaptive test or for a non-adaptive computer-based test.

LSAC has definitely considered eventually moving to an adaptive test. How do I know? I took one for the team and dove into the deep, dark section of LSAC’s website where none dare venture: the research reports. Creating a valid and secure CAT presents some challenges, and LSAC has studied how to overcome those challenges.

Since a CAT requires a large pool of questions, it’s unrealistic to write new ones for every test day. That requires re-using questions. How do you detect cheating if people somehow get access to some questions? They’ve studied that. How do you come up with the best, most awesome and comprehensive pool of potential questions? They’ve studied that, too. How do you give test takers their questions in the best possible order? Ditto.

Any change to the LSAT probably won’t happen quickly. Even if the LSAT goes computer-based, it might not be adaptive, at least not at first. But there’s a good chance the future of the LSAT is computer-adaptive. After all, having an adaptive test is one of the best reasons to administer it on a computer. What do you think? Would the prospect of a computer-adaptive LSAT scare you, or would you love a shorter LSAT even if you couldn’t skip around between questions? Comment away!

Leave a Reply

Your email will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>