When the scores for the most recent LSAT are released, test takers receive a series of documents: their score report, their question responses, the score distribution, a copy of their exam (unless you happen to take the undisclosed February exam), and the like. Most normal people, who see the LSAT as merely a hurdle in their path two a law school and a legal career, just look at their score and discard the rest.
But for the decidedly not normal and exceedingly nerdy — read: LSAT instructors — those discarded documents tantalize us. We love to pour over each released exam to analyze the questions, try to discern trends, and to predict changes in what is tested on the LSAT. We often compare each LSAT release to Christmas morning. We’re a very sad lot. But it was nonetheless very fitting that the December 2017 LSAT drop occurred so close to Christmas, giving us our favorite gift of all: the opportunity to gain insights into the LSAT that we can then pass along to our students.
So today we’re going to look over each section of the December exam to try to shed some light on some perplexing questions, to discuss which concepts this exam emphasized and downplayed, and to place this LSAT into the bigger picture of recent exams. If you took the December exam and are now hard at work filling out your application materials, hopefully this will be a revealing — and not too traumatic — walk down memory lane. If you’re studying for a future exam, hopefully this will be somewhat prescient in what your exam might look like.
So enough preamble, let’s do some quick hits on each section …
• The first thing I check out with Logical Reasoning sections is the question distribution. Although an LR section could conceivably discuss any topic — and there were some bonkers topics, on this and every other exam — there are only a few different kinds of questions they ask (about 18 common question types, according to our classification system). And the number of each type of questions included remains fairly constant over time, although there have been some changes on recent exams.
The biggest change to the question distribution for Dec 2017? There were a ton of Strengthen questions. By my count, there were 11 overall. I believe that’s more than any other LSAT, ever. However, this is of a piece with a larger trend on recent LSATs. Strengthen questions have become increasingly prominent over the last few years. So much so that they have recently overtaken Flaw questions as the most common question type.
• A few of these Strengthen questions were presented in an unusual way. Often, an LSAT question will ask you to select the answer choice that “most logically completes” the argument. You look up to the passage, and you see a big blank space following a word like “therefore” or “thus.” Those words tell that you’ll be “completing” that argument by providing the conclusion. Since you’re trying to infer what a supported conclusion might look like, those questions are what we call Soft Must Be True questions.
Several questions on the December exam asked you to select the answer choice that “most logically completes” the argument, but when you looked up to the passage, it used words like “because” and “since” before the blank space. That means that you’re adding a premise to the argument that will support the conclusion already included in the argument. In other words, these questions were asking you to pick an answer choice that would strengthen the argument, not draw a conclusion from that argument. These were Strengthen questions.
Your approach to Soft Must Be True and Strengthen questions are very, very different (for one, you should prefer weaker answer choices on Soft Must Be True questions and stronger answer choices on Strengthen questions), so realizing that these were Strengthen questions was pretty important.
Still confused on the difference between the two “most logically completes” questions? It’s simple. If the blank space follows words like “therefore,” “thus,” “hence,” or “so,” treat it like a Soft Must Be True question. If the blank space follows words like “since,” “because,” “for,” or “as,” treat it like a Strengthen question.
• Another interesting tidbit was the number of Disagree questions. The typical LSAT will have 1 or 2 of these questions. This one had 5. Plus, there were 4 Disagree questions on the September 2017 exam. So it appears as if Disagree questions are becoming a little more prevalent. And I, for one, see this as a welcome development. For a test that prepares students to enter law school — where they will learn about our two party, adversarial legal system and spend quite a bit of time pouring over old legal cases and describing the two sides of the suit — it makes sense that test takers will see several questions that test their ability to describe a disagreement.
• Otherwise, this was a pretty normal test in terms of question distribution. Other than Strengthen questions, Flaw, Necessary, and Soft Must Be True questions were the most common question types, as they pretty much always are. There were two Parallel and two Parallel Flaw questions, as always. There was only 1 Describe question, 1 Role question, and 1 Sufficient question. And there weren’t any Main Point, Crux, or Explain questions. But those are less common question types, so getting one or none of each isn’t unheard of.
• There were a few questions that lodged themselves in test takers’ memories following the exam, and, unsurprisingly, these were the most interesting questions to unpack. One question that stuck out for many — which has shot straight to my short list of favorite LSAT questions ever (again, see the above statement about LSAT instructors being a sad lot) — was a Soft Must Be True question about T. Rex bones. Apparently, some fossilized T. Rexes bear evidence of bite marks that could only come from another T. Rex trying to fight them or eat them. The question then told us that these bite marks would have been impossible to inflict on a live T. Rex. Which should have led the astute test taker to deduce that the bite marks couldn’t have come from live combat, but instead would have to come from a T. Rex mawing on the dead T. Rex’s rotting corpse. The right answer required a bit of common sense and outside knowledge — that eating dead kin constitutes cannibalism — but if you figured that out, then bang a gong and get it on to the next question, because you aced that one.
• The hardest question, I thought, was a Strengthen question about Caligula, the vilified Roman emperor, alleged freak, and subject of a very bad 70s film. This question asked us to support the position held by modern historians, who claim that Caligula isn’t as bad as he’s made out to be, since there isn’t that much documentation of his evil acts and the historical accounts of him were written by his enemies. Usually, getting a Strengthen question right is just a matter of figuring out what’s wrong with the argument. On this question, figuring out that part was easy. Just because enemies of Caligula described his bad behavior doesn’t mean that their descriptions were inaccurate. Befitting the Latin language Caligula spoke, that’s a classic ad hominem attack.
However, finding the answer choice that fixed that problem — by showing the enemies’ descriptions were inaccurate — was pretty tough. The right answer showed that these descriptions were inaccurate in a very oblique way. It said that the bad behavior attributed to Caligula is very similar to the bad behavior attributed to other cruel tyrants. That opens up the possibility that the enemies are either lying about the cruel stuff Caligula did (and using the older tyrants as their inspiration) or are confused about who committed the cruel acts (and misattributing them to Caligula). And either would buttress the position that Caligula — who reportedly engaged in incest and fed humans to wild beasts for pleasure — really wasn’t all that bad of a guy. Pretty damn difficult. What this question asked of you, at the end of the day, well, Caligula would have blushed.
• Aside from that question, how difficult was Logical Reasoning overall? Well after doing the exam, I assigned a number from 1 (least difficult) to 5 (most difficult) to each question. Not exactly the most scientific of methods, but I probably wouldn’t be doing this if I were good at math or science. Anyway, the average difficulty of each question was 2.68 for the first LR section and 2.65 for the second. So, there you have it, very average difficulty!
• My students’ reports following the December exam suggested that this Reading Comprehension section was uncharacteristically easy, especially after many, many exams of excruciatingly difficult Reading Comp sections. I held out hope that students just felt that way because I did an especially good job of teaching them the strategies of how to do this section. While I know our approach to Reading Comp is sound, my ego is reluctantly taking this L, because this was a pretty forgiving section.
• One noteworthy thing about this Reading Comp section? All but the last passage were organized around answering an explicit or implicit question. This is a common rhetorical device on Reading Comp passages, and identifying it can help you understand the passage and destroy on the questions. That’s because the main point of the passage will always be the answer the author agrees with (or a summary of each answer described, if the author doesn’t agree with any). So the first passage asked whether the “Chinatown Chinese” dialect from San Francisco’s Chinatown was a distinct Chinese dialect (Answer: Nope). The second passage asked whether life developing in our universe was really as improbable as some cosmologists believe (Answer: Probably not). In the third, comparative passage, passage A asked why comedians don’t use the legal system to protect their jokes (Answer: it’s expensive and uncertain, and social norms do the same thing) and passage B asked why chefs don’t use the legal system to protect their recipes (Answer: again, social norms do the same thing). By my count, just recognizing this would have directly answered or helped answer 13 of the 20 questions distributed among these passages.
• San Francisco’s Chinatown was discussed at length in the first passage, so let me use this space to offer my hot take. Aside from a few decent bars, San Francisco’s Chinatown is hugely overrated. Los Angeles’s Chinatown, which is smaller and not even the nexus of Chinese activity in the city, is much better. NorCal types, don’t @ me.
• The second passage is all about the theory of the multiverse — that there are an infinite number of universes, occurring parallel to ours, in which things played out differently than they did in our universe. The concept of infinite parallel universes is pretty wild if you think about it — literally every possible scenario exists in some other universe. Since LSAC brought that up, I think this opens up a great opportunity for you to write an explanatory essay on why your LSAT score isn’t up to some schools’ standards. Just explain, as calmly and rationally as you can, that although you didn’t get a 180 on this LSAT, there is a version of you in some other parallel universe that earned a 180 on the LSAT, and that the admissions officers should take that into account. Hey, if they’re going to take past scores into consideration, they should take parallel scores into consideration as well.
• The final passage illustrated an important lesson many test takers unfortunately fail to learn for Reading Comp. So many test takers spend way too much time trying to understand every last detail of the passage, and don’t give themselves enough time to answer the questions. Now, understanding the structure of the passage and the author’s attitude are the most important tasks in Reading Comp, and you should take adequate time to do those. But you don’t have to understand every last little detail.
Parts of the fourth passage, about how a novelist and social theorist named Charlotte Perkins Gilman took her understanding of Social Darwinism and used it to support a feminist theory of human progress, were confusing as hell. You could have easily wasted 10 minutes on them. But the questions, I thought, were super easy, and really just depended on you understanding which camp of Social Darwinists Gilman belonged to. So remember: focus on the structure and author in Reading Comp. If you need the details of a passage, you can always re-read the relevant parts of the passage.
• Using the 1 to 5 scale for the Reading Comp section, the average difficulty was a 2.75. That’s a half-point lower than the September 2017 section and a full point lower than the June 2017 section. Is this the end of brutal Reading Comp sections? I wouldn’t bet my life’s savings on that. There is still far more evidence of tough Reading Comp sections to come. But we’ll see February!
• Finally, logic games. After a few exams of straightforward games with straightforward deductions, which followed a few years of the LSAT getting a little bit wild with some of its games, December 2017 gave a set of straightforward(-ish) games with straightforward(-ish) deductions. I’d place the difficulty of this section somewhere between the notoriously easy September 2017 section and some of the really difficult recent sections, like say, December 2016.
Now, I heard many frustrated test takers who found this to be a really difficult games section. I understand where they were coming from, as these games were decidedly more difficult than the games they did on their final practice exams, assuming they took the September and June 2017 exams last. But these weren’t really mind-melting or unfair or even usual games. I suspect a big reason so many found this lot of games difficult was that this section was the last of the exam, and students had to tackle them while fatigued.
• The common element to each game was making scenarios. Every game benefitted from the use of scenarios. If you made the right set of scenarios for each game, they were actually really easy (I completed all four games in about 20 minutes). Unfortunately, none of these games included the rules that obviously lend themselves to scenarios. Like a giant constrained block in an ordering game or a “must be together” relationship in a grouping game. So test takers had to get a little creative to find ways to construct scenarios. But those ways were definitely there.
• The first game was a “combo” game — combining an ordering game with an In & Out grouping game. Basically, a travel agent (which, btw, is a profession that’s antiquated even to me, who is part of an age group that could generously be referred to as “Washed Millennials”) has to select four of six Asian cities for a tour, and then schedule the selected cities from the first to fourth city traveled to.
The key to constructing scenarios in this game was realizing that, since Hanoi and Taipei were cities that must be included in the tour, the only cities that could fill out the remaining two open slots were Jakarta, Manila, and either Osaka or Shanghai (a rule forbid selecting both). Selecting two of three is always a potential way to make three scenarios, and that was the route to take for this game. Making one scenario where the last two cities were Jakarta and Manila, one where they were Jakarta and either Osaka or Shanghai, and one where they were Manila and either Osaka or Shanghai answered every question.
• The second game was a straightforward 1:1 Ordering game that used a difficult, but common, ordering rule: that a player had to be either before or after two other players. Rules that present two mutually exclusive options are pretty much always good for two scenarios. These scenarios combined nicely with another rule that presented two mutually exclusive options — that a player was either before or after another player — to create scenarios that helped out with each question.
• The third game was a source of consternation for many test takers, with many failing to figure out what type of game it was. The game involved a passenger railway system needing to close some of its six stations. So some stations would be left open, and some would be closed. Some open. Some closed. Open/Closed. In/Out. “Open” and “closed” may be the binary, but it’s still not all that different from an “in” and “out” binary.
This is an important lesson — a game may seem strange at first blush, but odds are that it fits in the paradigm of a safe and familiar game. And you’ll be so much better off doing that game using the same set-up and rules as a game you’ve down dozens of times before. For those studying for the next exam, btw, this was very similar to game 2 in the October 2005 exam (in which light switches are either turned “on” or “off”) and game 1 in the 2008 exam (in which dancers are either “on” or “off” the stage”).
Once you figured out this was an In & Out game, where the “In” group was stations left open and the “Out” group were stations that were closed, scenarios presented themselves with the first rule: that either N or R had to be “In.” That restriction combined with other rules and, by making the deductions in each scenario, the clever test taker should realize that N and R couldn’t both be “In.” There were many unrestricted players in this game, but the scenarios still provided a framework to quickly answer every question.
• The fourth and final game was admittedly difficult. Again, many test takers struggled to even set this up. Here’s an easy tip for setting up games, though: scheduling is an act of ordering. So whenever you get days of the week in a game, it’s an ordering game. So in this game, we had to schedule a day, Wednesday through Saturday, for an environmental consultant to examine the air quality on floors 1 through 8 of a building. The consultant would examine two floors each day, making this an Overbooked Ordering game.
One hard part of this game was the first rule, which prevented the consultant from examining consecutive floors on the same day. Seems inefficient, but sure, do your thing environmental consultant. Anyway, I suspect many got hung up on symbolizing this rule. But it’s more important to understand and apply the rule. This rule just means that if, say, floor 4 is examined on Thursday, floor 3 and floor 5 couldn’t also be examined on Thursday.
A pretty unique set of scenarios emerged in this game. The game placed a series of mutually exclusive options on certain floors. Floor 3 could only be examined on Wednesday or Thursday. Floor 4 could only be examined on Thursday or Friday. And Floor 5 could only be examined on Friday or Saturday. As on the second game of this exam, these mutually exclusive options were the basis of scenarios. These constraints, plus the first rule, which prevented floors 3 and 4 and floors 4 and 5 from going on the same day, gave us four scenarios: One were 3 was on Wednesday, 4 on Thursday, and 5 on Friday; one where 3 was on Wednesday, 4 on Thursday, and 5 on Saturday; one where 3 was on Wednesday, 4 on Friday, and 5 on Saturday; and one where 3 was on Thursday, 4 on Friday, and 5 on Saturday. Again, these scenarios helped on pretty much every question.
• Hopefully the above descriptions make clear one point about learning how to do logic games. Practice making scenarios! It can even be worthwhile to do games multiple times — once with scenarios and once without. Experiment on each game, and see which approach is better. This will help you begin to recognize which types of constraints are most useful for scenarios, and will help you choose the most efficient path for each game on your exam.
• Here’s a final point about games: If you’re reading this blog (and made it this far into the post), then you clearly take your studies very seriously. Bully to you! Hopefully that means you’re practicing more than the typical test taker. If so, you should want a difficult games section. Nay, you should crave a difficult games section. You’re going to be better equipped to handle more complex or unusual games than others. Take it from a guy who has taught hundreds of people — most fear difficult games more than anything on the exam. However, with the right practice, difficult games are the most conquerable “difficult” part of the LSAT. On the easier games sections this year, most people did very well on logic games. To distinguish yourself from the crowd and get a good score, you had to do really well on a much more difficult to master Reading Comp section. Trust, it’s a lot easier to distinguish yourself on a difficult games section. So don’t fear difficult games. You can master them. And then tests like December’s will be all the more conquerable for you.
• So in all, this wound up being a slightly more difficult test than September or June 2017, but with a slightly more forgiving curve. For the December exam, you could miss 12 questions and still get a 170. This is much more generous than the 9 you could miss on the June 2017 exam and a little more generous than the 11 you could miss on the September exam.
• It also caps off a year when the LSATs were all fairly straightforward and predictable. At least more so than usual. Once 2018, with its slightly revised schedule of LSAT administrations, begin to roll around, we’ll see if this old dog of a test still has some tricks up its sleeves, or if it’ll be more of the same.