Impressions on the new practice PSAT

A few days ago the College Board posted a new PSAT practice test.  This is the practice test for the redesigned PSAT to be given in October 2015.  It is also a glimpse of what the redesigned SAT might look like. 

There has been a great deal of speculation about the redesigned test.   A number of people have opined that the new test would be an “ACT clone.”  I, myself, have speculated that the new test would be designed as more of a high school exit exam than a college entrance exam.  Quite a few pundits have pointed out that the SAT was losing market share to the ACT and thought that the redesign might be an effort to gain that share back.

Upon looking over the test, I am now prepared to make the following statements:

The writing section IS barely distinguishable from the writing portion of the ACT.  Otherwise, this test is in no way an ACT clone.  On the other hand, I do feel that this test represents a fundamental shift in purpose.

There are a number of possible purposes for giving a standardized exam.  (This would be as opposed to a teacher-made assessment.)  Here are a few:

• To document student learning (or lack thereof) of particular skills and concepts
• To distinguish among (or rank) students
• To drive the curriculum*

Prior to 2015, the PSAT’s fundamental purpose has been a balance of the first two items.  As the National Merit Scholarship Qualifying Test it has been used to distinguish top students from the pack, and it has documented whether or not students have a grasp of particular skills and concepts.  It has NOT been used to drive the curriculum...until now.  In my opinion, several of the changes were specifically designed to have an impact on the nature and content of classroom instruction.

One of the elements of Common Core language arts is a focus on having students read critically.  In an effort to have the students respond to what the author actually said – as opposed to how they feel about what they think the author said – students are being asked to point to pieces of the text as “evidence.”  In the PSAT critical reading portion, students are asked to choose the best evidence for almost every question.  Now I’m not a language arts teacher, and I don’t have any special critical reading or psychometric expertise.  However, it seems to me that this is about the same as asking the same question twice.  In other words, if you correctly answer the original question, then the answer to the “evidence” follow up is trivial.  If you missed the original question it would be impossible.  Will there be some kind of scoring mechanism that uses this question-pairing to determine when the correct answer was obtained by guessing?  Maybe.  It is more likely an attempt to make sure teachers require their students to give evidence in class.

Here is another example from the math section.  Months ago, when the sample questions were released, I noticed that some of the questions were much longer and more involved.  This represents a distinct shift.  Up until now I have told my top students, “If you are more than 3 steps into an algebra process, you probably missed something.”  One of my main criticisms of some of the test prep books out there has been that too many of the math questions can ONLY be solved with the application of tedious algebraic steps and are thus not representative of the real thing.  However, this has changed with the PSAT practice test.  It’s interesting.  If I’m a professor of mathematics or engineering I’m interested in multiple different aspects of their math skills.  I want them to have a solid grasp of the concepts, good number sense, AND I want them to be able to keep track of what they are doing through a long problem that requires many steps and sub-steps to solve.

Up until now, the major college entrance exams did a decent job of testing the first two. (The SAT was better than the ACT in my opinion.) They really haven’t attempted to do the third.  This is largely because a multiple choice or single final answer exam format is a TERRIBLE way to assess that third skill.  If the student’s answer is incorrect, you don’t know if he really can’t negotiate the process or if he just made a silly error in the middle.  Graded homework and teacher-made assessments where partial credit is given are much better means of determining whether or not the student can handle a long problem.  Both of those would be reflected in the students’ grades.  By trying to test it in this format, you add no new information.  The only reason I can come up with to include problems like this is to encourage math teachers to have students practice longer, more complex problems.

When the College Board first announced that the SAT would be re-designed, an admissions officer at a small, selective school wondered in an online forum, “Will the rSAT do a better job of distinguishing among students at the top end of the spectrum?”  This was what she was hoping for.  Other users of the forum predicted that it would not.  To do so would require a test with a wider standard deviation, and a large contingent of the (math-challenged) public believes that a wide standard deviation is inherently “unfair.”  In fact, the speculation was that the new test would do a worse job of highlighting differences among students.  Given that the redesigned test appears to have abandoned that goal altogether in favor of driving the curriculum, I’d say that admissions officers at selective schools will be plumb out of luck.

The practice PSAT is posted here: https://collegereadiness.collegeboard.org/sat-suite-assessments/practice/practice-tests

* You may be wondering what it means to have a test “drive the curriculum.”  Let’s look at an extreme example.  Throughout the 1990’s and early 2000’s North Carolina had a state writing assessment.  In fourth, eighth and tenth grades students had to write a timed essay and send it off to be scored by specially trained “experts.”  After several years of dismal results, some statisticians called “foul.”  They pointed out that the scoring methods were severely flawed, and thus that the scores were ultimately meaningless.  (By the way, the scoring methods used to score the ACT and SAT essays have some of the same issues.)  The state’s surprising response?  “Yes, we know.”  Students (and by extension their teachers and parents) suffered through this test for YEARS.  Why?  Because if there’s a writing test – even a flawed one - teachers will spend time teaching writing.  The average amount of classroom time spent on writing instruction quadrupled.  Prior to the test, some teachers had spent ZERO time teaching writing.

To pre-order the practice book for the redesigned SAT:

The new SAT: the Essay

This is the third post in a series on the redesigned SAT.  The first two were on vocabulary and the math section, respectively.

It was interesting to see what the news media chose to focus on when they did their bullet points for the redesigned SAT.  As headlines and/or sound-bites, many chose variations on “The new SAT will drop the essay requirement!”  That they would trumpet this particular change is understandable.  The SAT essay has been widely loathed since it was first introduced in 2005.

Criticisms of the 25-minute essay section abound:  Writing a coherent essay in such a short period of time on a random prompt that was sprung on you at the last second is a very artificial task.  How often does that come up?  Even college blue-book exams – the closest real-world situation - will have essay questions specifically on the course material.  And don’t get people started on how they are scored.  Anecdotes abound about how kids who can afford coaches are at an enormous advantage because coaches teach you how to use really bad writing to get a top score.  Dropping the essay requirement was popular and a good marketing move.

However, "drop the essay requirement" may be misleading.  The SAT still has an essay.  However, now the style of essay has changed and it is "optional."  Changing the style of the essay is a good move. The new style is much more the kind of writing that you might expect to do in the workplace:  Take this data and write a quick position summary.  ETS has already test-driven this essay style on the GRE, so they should be able to implement it on the SAT with few hiccups.

As far as being "optional" goes, the more selective colleges will "require" it, just as they now "require" the "optional" ACT essay.

While it is true that the method of scoring the essay has always had issues - and likely still will - there are two points to consider here:  First, American kids need to be writing more and when you include essays in assessments, the curriculum will include more writing instruction.  Second, colleges have always been able to read a student's actual essay.  They don't have to rely on the score as a measure of an essay's quality.  In fact, the essay score has always had only a slight impact on the students Writing component score in addition to being reported as a separate score.  Many colleges disregard the entire Writing component score altogether while others downplay it.  Yet, they have required the essay anyway. Keep in mind that this will probably be the only sample of writing accessible to the admissions committee that is guaranteed to have been written by the applicant.  Admissions officers all claim they can spot an adult-written essay from a mile off, but can they really?  And what is to prevent a student from paying his classmate to write a better essay?

So the essay is still there and those applying to selective colleges are stuck with it.  However, the task should be a better example of a real-world writing situation, and practicing for it may pay off even after the SAT is over.

When it comes to punctuation, less is more

In my last post, I mentioned that I had taken a practice ACT writing section without reading the passages.  I used certain rules for deciding which choice to answer that take advantage of the odds of particular types of answers being correct.  If the question was testing punctuation, the rule was that I had to choose the answer with the least punctuation.  I did this even in cases when the choice was obviously incorrect even within the answer choice.  For example, if one of the choices was "the teachers classroom"  I would have to choose it (no punctuation obviously being the least punctuation) even though the correct choice clearly needs an apostrophe.  If two choices tied for least punctuation I had other methods (such as "odd man out") to break the tie.

Among the punctuation questions, my choice was correct about 40% of the time - far greater than one would expect from random guessing.  When my choice was incorrect, the choice with the next least punctuation would have been right about 40% of the time.  The choice with the most punctuation was almost never right.

Now, obviously, it is best to know your punctuation rules and to choose your answers based on which choice you know to be correct.  However, if you must guess from among two or more choices, you should go with the odds and choose the one with the least punctuation, or, at the very least, eliminate the choice with the most punctuation.

ACT English: The Odd Man Out "trick"

"One of these things is not like the others. One of these things doesn't belong...."

This was a song on Sesame Street back in the late 1960's.  Sometime between then  and the 1990's someone decided that this was not a good educational activity and Sesame Street discontinued it.  When I sing it to today's teenagers they look at me funny.

(You can see a Sesame Street clip of that activity here.)

It tends to come up when I am coaching someone for the ACT English section.  The correct answer is frequently the "odd man out" - the one choice of the four that is grammatically different from the other three.

For example, in one question from The REAL ACT Prep Guide the answer choices were:

a.  NO CHANGE  (it was written as "straighten it" in the passage)

b.  straightened them

c.  straightened those

d.  straighten them

Choice "a" is the only one in which the pronoun is singular instead of plural, and choice "a" was, in fact, the correct answer.

I have noticed a number of trends about the frequency of correct answer choices on this section of the ACT, so I recently did an experiment.  I completed an ACT English section without reading the passages.  Instead I chose an answer based on a number of rules.  One of the rules was that if there were an identifiable "odd man out" I would choose it.  For each question I wrote my answer choice and then next to it the rule I had used to choose my answer.  I used the "odd man out" rule 20 times.  The choice was correct 11 times, or 55% of the time.

I know that the anti-test campaigners are routinely horrified by test strategies that allow one to answer a question without even reading the material, but I would like to point out that this particular strategy does require a knowledge of grammar.  One must recognize that one of the choices is different in a particular way.

I am rarely asked to coach for the ACT (the SAT is a bigger deal in my state,) but I will continue to put considerable emphasis on recognizing when "one of these things is not like the others."

Study materials review: Up Your Score

Up Your Score:  The Underground Guide to the SAT has been around so long it is no longer "underground."  First published in the 1980's, each edition stays fresh by recruiting top-scoring students as co-authors.  The 2013-2014 edition is by Larry Burger, Michael Colton, Jaja Liao, Manek Mistry and Paul Rossi.

This guide is intended as a supplement to The Official SAT Study Guide.  Up Your Score does not include any practice exercises.  Compared to other supplementary guides, this book includes more advice on studying (how to concentrate, how to improve your memory, best time of day to memorize something, etc) stress reduction, and general test-taking strategies.  The section on guessing strategies is one of the most interesting of its type that I've read. There is even a section on bubbling technique.  Apparently some students take a ridiculously long time to color in the bubbles. Who knew? Most of the tips are of the type you see in every guide, but there are a few new ones here. This guide is more fun than most.

When it comes to content, there isn't much here.  One third of the book is devoted to vocabulary.  The book doesn't have a whole lot to say on the rest of the critical reading section.  There are 600 words in the vocabulary list.  The bad news:  they are arranged alphabetically instead of by meaning.  The good news:  after dozen words or so there is a silly alliterative story using those words designed to help you remember them.  Another thing this book does well is to distinguish between words you might need to know to answer the vocabulary questions at the beginning and literary terms you might need to know to answer the passage content questions.

The math section begins on such an elementary level that, honestly, if you don't know how to do that then you aren't the sort of person who can learn math from a book.  It then proceeds to brush lightly over far too much material without either giving an opportunity to practice or referring you to appropriate problems in The Official Guide.

The writing section is short and tries to cover both the multiple choice content and essay advice.

I picked up a copy at BJ's for under $10, so the price isn't all that bad.  However, if you've already read another guide, this one doesn't have enough new stuff to make it worth buying.  Check a copy out at your school or local library and then read just the useful bits.  If this will be your first guide it's worth the price, although if you find out that you need more intensive help in a particular section, this one won't cut it for you.

Study Material Review: Barron's ACT 36

My idea of an ideal ACT 36 book is one that accomplishes two things:

1. Discusses how to answer the most difficult ACT questions and gives plenty of practice for each type.
2.  Shows the student how to save time on the easier portions of the test so that there will be enough time to work on the most difficult questions.

Barron's ACT 36 does neither of these things.  In particular, there are not nearly enough practice problems. 

Use a $5 word in your essay.

Have you ever heard the saying "Don't use a $5 word when a 50¢ word will do?" This does not apply to your entrance exam essay. One of the things the scorers look for is your use of vocabulary. This is not to say that every word needs to be the most sophisticated word you can find. You don't want your essay to sound like you wrote it with a thesaurus in your lap.  Besides, 40 or 50 minutes isn't enough time to write an essay in which each word is the best possible word for that space.  However, during the last minute or so when you have - hopefully - finished writing, read over your essay. If you haven't used any college-level words, then pick one word and replace it with a better one. Let the scorers know that you are comfortable with college-level vocabulary.

Don't dismiss the shortest answer!

Don't dismiss the shortest answer without a reason!

On the writing sections of college entrance exams, any form of wordiness or redundancy is considered an error. Therefore, shorter answers are somewhat more likely to be correct. If you have the choices narrowed down to two and you can't choose between them, pick the shorter answer.