your connection to The Boston Globe

New ranking system based on choice

Critics of college rankings say universities aren't like their sports teams. You cannot settle who is best with head-to-head competition on the playing field.

But in a new paper, a group of economists and statisticians begs to differ.

They lay out a system that ranks colleges on how they perform in one kind of head-to-head competition they contend says a lot about a school and can be measured: the battle for students who are admitted to several colleges.

While the US News & World Report rankings use statistics like admission percentage, Scholastic Assessment Test scores, and student-faculty ratio, critics say that those figures are not necessarily much use to prospective students and that colleges can manipulate them.

In their proposal, the economists sidestep the tricky question of what makes a good college. Instead, they assume top high school students know best, and they simply report their choices. Of the students admitted to, say, both Brown University and the University of Pennsylvania, how many choose each place? It is the same principle in the Zagat restaurant guides: Don't try to grade the food, just say whether a lot of people like it or not.

The authors -- Caroline Hoxby and Christopher Avery of Harvard, Andrew Metrick of the University of Pennsylvania, and Mark Glickman of Boston University -- have been working on their model for years. Their most detailed results yet were published recently by the National Bureau of Economic Research.

Here's how it works: Imagine two students, one choosing between Stanford and Harvard, another among Stanford, Berkeley, and Pomona. The statistical model views each one of those students as a ''tournament" between the colleges involved; if Stanford ''wins" either student, its ranking rises in relation to the schools it beat.

Colleges often compete against the same schools over and over and may never compete against others. But with enough data, Stanford's place in relation to all schools begins to emerge and the rankings take shape.

The model, which resembles the one used to rank professional chess players, adjusts to balance out influencing factors such as a big financial aid offer from one school.

The authors offered a preliminary demonstration by tracking the college choices of 3,240 high-performing students from 396 high schools nationwide. They say that it works well for top schools but that more data are needed to improve their confidence in the rankings lower down.

So what does their trial run show? The top 20 schools look similar to the top liberal arts colleges and universities in the US News rankings (the categories are combined in this study), but their order changes. In US News, for example, Harvard and Princeton share the top spot among national universities; here, Harvard is first, while Princeton falls to sixth. Duke, fifth in US News, drops to 19th.

But the system rewards other schools. Wellesley's appeal as a women's college evidently helps it beat elite universities. Georgetown and Notre Dame score higher than they do in US News, probably because of their popularity with Roman Catholic students.

In an interview, Hoxby said high school students have a good sense of what makes a good college. And she said colleges cannot game this system.

Many critics of college rankings contend that some schools try to lower their admissions percentage -- and make themselves look more selective -- by encouraging applications from students who have no chance of getting in. Or they try to boost their matriculation rates -- the percentage of admitted students who enroll -- by turning down top applicants they suspect are just applying as a backup and are not likely to come.

But under this new ranking system, the only way for colleges to improve their position is to get more top students to apply and attend.

''Nobody would feel under pressure to manipulate their admissions and matriculation rates," Hoxby said.

Facing criticism, US News & World Report recently dropped matriculation rate as a component of its formula.

Robert Morse, the magazine's director of data research, said admissions percentage plays a tiny role. As for the economists' approach, Morse said he doubts they could get the expanded data they need to make the system more credible.

Today (free)
Yesterday (free)
Past 30 days
Last 12 months
 Advanced search / Historic Archives