Vassar questions validity of U.S. News and World Report college rankings

Brian Farkas News Editor

Vassar College President Catharine Bond Hill announced this week that Vassar, along with 18 other liberal arts colleges, will no longer mention the U.S. News & World Report (USNWR) rankings in any future promotional publications. They will also make public all statistics that were formerly distributed to individual ranking companies.

“Such lists mislead the public into thinking that the complexities of American higher education can be reduced to one number,” read a joint statement posted Sept. 7 on Hill’s Web page. “We are concerned about the inevitable biases in any single ranking formula, about the admissions frenzy, and the way in which rankings can contribute to that frenzy and to a false sense that educational success or fit can be ranked in a single numerical list.”

The colleges that signed onto this agreement include all but two of the top 15 liberal arts schools in the nation, as ranked by USNWR. The list includes peer institutions such as Williams College, Amherst College, Swarthmore College and Wesleyan University.

The schools believe that making data and information public will increase transparency and decrease the emphasis placed on a single ranking system. “Doing so is true to our educational mission and will allow interested parties to use this information for their own benefit. If, for example, class size is their focus, they will have that information. If it is the graduation rate, that will be easy to find,” read Hill’s statement.

Late this August, USNWR issued its annual rankings of colleges and universities. Vassar moved closer to the top of the liberal arts college rankings this year, resting at no. 11. It is tied with Claremont McKenna College, Grinnell College and Wesleyan University. As in 2007, Williams College topped the list, followed by Amherst College and Swarthmore College.

The lists, often considered the gold standard of comparison between institutions of higher learning, have come under fire in recent years for the way they determine the top-ranked schools.

Hill, an economist who specializes in higher education, is critical of the statistical methodology and assumptions that underlie USNWR’s system. “The ranking is an index of a set of variables. It would be a mistake to take it as an objective measure of the quality of the schools it ranks,” said Hill. “There is no consensus on what variables or weights to use, and in fact, this will differ for different students.” This July, Hill published an article on the subject in The Christian Science Monitor in which she called the USNWR rankings an “annual academic beauty pageant.”

The ranking takes various data and then weighs the variables to give each institution a unique ranking. That final number, though, reflects only the relative weight given to each piece of data. In other words, the rankings reflect USNWR’s preferences. However, if certain statistics such as class size, SAT scores, or endowment matter more or less to a student, then the USNWR list does not accurately reflect his or her preferences.

“The point is that there is no consensus on what variables should be in a ranking and how much weight to give each variable,” said Hill. “And there shouldn’t be. It will differ for each student. Students have different preferences and different needs, so searching for a unique, objective ranking of schools borders on being silly.”

Hill hopes that Vassar’s “moving up” from no. 12 to no. 11 will not be a significant factor for any applicant. “We went from being tied for 12 with one other school to being tied for 11 with three other schools. I hope no one lets that influence his or her decision about where to go!”

Dean of Admissions and Financial Aid David Borus agreed. “Most students use USNWR as a jumping-off point. The level of specificity between one or two or even five places can be very arbitrary,” he said. Borus said that he has seen many ranking systems in his 30 years at the College, and doubts that it affects the “quality of applications Vassar recieves.’

Despite her contention with the survey, Hill acknowledged that the USNWR rankings have some benefits for prospective students. The magazine publishes some important statistics about a large number of institutions. “This information has not always been readily available,” she said. “Schools, partly in response to the USNWR, are trying to be more forthcoming with information that they believe matters.”

Hill also does not believe that pulling out of the rankings is the best option for Vassar. “At the moment, there is some suspicion about higher education on the part of the public, for a variety of reasons. To stop providing information seems like the wrong thing to do. It would suggest we have something to hide,” she said. In fact, no school can elect not to be ranked—it can only elect to stop providing USNWR with information. The magazine can easily find most of those statistics even without the participation of the institution, and will rank them regardless. Colleges that receive federal funding must report a variety of data to the Department of Education.

“Students should decide what they care about and then look for the school that best meets their needs,” Hill added, “The USNWR is not going to be able to do this for them.”

In the last few years, much of the academic world has been abuzz with criticism about the validity of the USNWR system. Later this month, dozens of higher education officials will hold a conference at Yale University to discuss plans for alternative ranking systems that will be based more on individual student preferences.

A major factor in the computation of the USNWR rankings (25 percent, to be exact) is a reputation survey in which colleges rank their peer institutions. For the 2000 issue, some 60 percent of colleges and universities provided information for the survey; for the 2008 issue, only 51 percent elected to contribute information to the publication, the lowest participation rate in the survey’s history.

Hill completed the survey this year, and will continue to do so. “I actually think presidents of colleges and universities do know something about other colleges and universities and can add some information to the other available data,” said Hill. “Again, does it lead to a unique ranking? No, it just tells you what college and university presidents think about the schools. As a student, you might find that interesting, or you might decide it didn’t matter to you.”

Hill’s July Christian Science Monitor article proposed ways to minimize the significance of USNWR. One of these suggestions, similar to the action commanded by last week’s statement, was to make public all of the information formerly sent only to USNWR. This would make the data free and available to students and families without any sort of ordinal ranking. Hill also hoped for a program that would “allow prospective students and their parents to essentially build their own rankings.”

But few people think that USNWR rankings will fade away so quickly. Director of Career Development Mary Raymond believes that rankings—and USNWR specifically—play a major role in many employment and graduate school admissions decisions.

“It might seem like a very arbitrary way of going about employment, but the fact of the matter is many employers—particularly larger companies and banks—use these lists to gauge their applicants,” she said.

Raymond sees rankings as one of the primary tools potential employers use in making hiring decisions, relying on these kinds of rankings to weed through applicants.

“It may not be the politically correct answer these days, but from where I’m sitting, it’s fact,” Raymond said. “I don’t pretend this is the right way to go, but if my job is to advise students on careers, I would be lying if I told them that Vassar’s position relative to its peer institutions was inconsequential.”

Graduate programs, though, on which the Career Development Office also advises, do not take the numbers quite as seriously. While rankings still give these admissions offices a general sense of an applicant’s education, Raymond believes that they are far more interested in a student’s specific achievements. In addition to GPA and GRE scores, a student’s work on a thesis or on independent research is far more important than a few slots on the annual rankings.

Director of Fellowships and Pre-Professional Advising Lisa Kooperman also believes that while the rankings are not perfect, they do prove useful and accurate in some respects. “I think it is safe to say that colleges that are in the top segments of their categories are, for the most part, schools with national recognition. Vassar is clearly well-known throughout the academic and work world. However, to say that a 10 versus an 11 or any other fine distinction makes a huge difference may be just a matter of opinion.” Kooperman argued that the same is true for their rankings of graduate schools.

In terms of admissions to these graduate programs, Vassar’s prestige on USNWR is only part of an applicant’s success.

“In general, I think that once a school has a national reputation such as Vassar’s, the emphasis by graduate, professional school, and fellowship selection committees turns much more to the individual applicant. It is other aspects of the applicant’s profile that will be defining,” said Kooperman.

The approach taken by the 19 liberal arts schools is not as drastic as the position taken by many other institutions in recent years, who have refused to cooperate with the USNWR rankings entirely. In the early 1990s, the Annapolis Group, a loose association of liberal arts colleges, lead over 40 schools to pull out of the rankings. These schools included Barnard College, Sarah Lawrence College and Kenyon College. The Annapolis Group members also decided to develop their own system of comparing their data publicly, without imposing any type of ordinal ranking.

Borus holds that the method taken by Vassar and the 18 other schools is fair. “Our approach is a reasonable and sensible one,” he said. “This will result in more information being made more public to a larger number of people. And best of all, students and families will be able to interpret this data for themselves.”