Friday, September 12, 2014

A less-biased look at Northeastern University's "meteoric" rise in the U.S. News and World Report ranking.

College rankings have taken a big PR hit in the last year.  Some of it is deserved.  However, as has become the norm in American public discourse, people eager to jump on the “bash the latest unpopular thing” bandwagon have demonstrated a remarkable lack of critical thinking skills.  This is especially sad when the topic is higher education – an institution that should be dedicated to encouraging critical thinking.

An article recently published in Boston Magazine by Max Kutner purports to be about how Northeastern University, located in Boston, managed to rise in the rankings of the U.S. News and World Report by “gaming” the system.  Before going on, it might be useful to note that “gaming the system” is typically defined to mean manipulating the rules in such a way as to gain an advantage.  It is generally understood that the entity “gaming the system” is not breaking the rules.  Rather, the entity is typically following the letter of the rules but not the intent.  Breaking the rules would be subject to disciplinary action of some kind.  “Gaming the system” generally is not.  Despite that, it is also generally understood that “gaming the system” is an unscrupulous act designed to obtain an advantage unfairly.

The article opens with a description of the state of Northeastern University in the early 1990’s.  Their situation was dire:  the school was under-enrolled and under-funded.  There was a real danger that if they could not turn things around, they might have to close their doors.  Enter one Dr. Richard Freeland who is charged by the author with making “gaming the U.S. News ….part of the university’s DNA.”

Here is a list of the things the university did under Freeland’s administration that resulted in a rise in the school’s ranking from 162 to 98:

  •       Reduce class sizes
  •        Begin accepting the Common Application, which made it easier for students to apply
  •        Constructed new dormitories because studies showed that student who lived on campus were more likely to graduate
  •        Do some PR to boost the school’s image
  •        Report the number of students each year differently to reflect the number of students on campus instead of including co-op students


Wow.  How nefarious of them.  The author emphasizes that Dr. Freeland kept his eyes on the rankings throughout the improvement process.  What he fails to acknowledge is that, while the college rankings are far from perfect, they do include some measures that legitimately affect the quality of education.  Smaller class sizes are not only linked to better learning outcomes, they are also a measure that potential students and their parents find interesting.  Surely no one thinks that making the application process more convenient and accessible is a bad thing.  And if graduation rates needed to be raised (and it seems they did) then building dormitories sounds more like “data-based decision-making” than “gaming the system.”  Oddly, the author carefully avoids telling us how much the graduation rate rose, but rise it must have – the subsequent increase in ranking could not have been obtained otherwise.  On what planet is that a bad thing?

The one item in the list that sounds like it might be shady is the last bullet point in the list.  Northeastern changed their reporting methods.  Dr. Freeland realized that the metric being used by U.S. News hurt schools with strong co-op programs.  Northeastern counted significantly more students each year than were actually on campus, which made it look like the school was spending a lot less per student.  He took his case to the U.S. News statisticians who declined to change their metric, but who explained what they were doing with the numbers and why.  As a result, Northeastern stopped including co-op students who weren’t on campus in their numbers.  Is that dishonest?  I don’t think so.  I think it makes for a more accurate picture of their situation.

The article includes a list of schools caught flat-out lying on the numbers they report to U.S. News.  While the author acknowledges that this does not fall under the category of “gaming the system,” he does offer this as evidence that the rankings are irretrievably broken – an accusation the magazine denies.  Including them tends to – intentionally or otherwise – give the impression that the measures Northeastern has taken are as dishonest as these examples.

Not until the last few paragraphs do we find any evidence of actual “gaming,” and these were introduced after Freeland retired in 2006.  Northeastern stopped requiring foreign students to submit SAT scores.  Foreign students, for whom English is often a second language, can have lower SAT scores.  Not to require scores from foreign students may be a bit shady, although it has recently come to my attention that taking the test represents a true hardship for many foreign students by requiring an overnight trip to a distant city.  Some might consider dropping the requirement an effort to be more understanding.  Then in 2007, the school began a program whereby students could begin at NU in the spring – thus excluding their data from the reporting.  The author states that these excluded test scores and GPA’s are “lower” but offers no evidence for this statement.  They certainly could be, and if they aren’t, one wonders why NU would begin the program.

As a final jab, the author points out that the measures taken to improve educational quality at NU – and quality has undeniably been improved by increasing retention and graduation rates, if nothing else – has cost money, making the school more expensive.  This is undoubtedly true, but it’s an odd accusation to make.  Typically the complaint is that we don’t spend enough on education, or that when we do spend more, we don’t see an improvement in results.  Here is an example of a school that spent more – and it paid off.  The alternative was to close their doors.  Does anyone wish to argue that they should have chosen that as the more honorable course of action?  The price increase does, indeed, make Northeastern one of the more expensive options out there, but price is one of the factors every family should weigh in making decisions about where to send a student to school.


In over 30 paragraphs of writing, the author only mentions 2 possibly unscrupulous methods Northeastern may have used to improve their ranking.  He mentions several instances in which the school used metrics in the ranking to guide decisions that led to improved outcomes.  At one point the author quotes Lloyd Thacker as saying, “Have rankings contributed to anything beneficial in education?  There’s no evidence.  There’s lots of evidence to the contrary.”  As a refutation of that statement, I would point to Northeastern University.