Colleges That Get it Right

Washington Monthly. Posted August 25, 2005.

This month, U.S. News & World Report releases its annual rankings of colleges. First published in 1983, the guide has become its own mini-event: College presidents, education reporters, alumni, parents, and high school juniors alike all scramble to get their hands on the rankings. Its release is followed by weeks of gloating from the top-ranked schools and grumbling from those schools that dropped a slot (or 14) from the previous year. Inspired by the popularity, other guides -- from Princeton Review to Peterson's to Kaplan -- have rushed to compete.

College rankings are now so influential that universities and higher-education journals hold regular chin-stroking sessions about whether the numbers-game has too much influence over the way schools behave. New York University's Vice President John Beckman sniffed to the Harvard Crimson this spring that the rankings "are a device to sell magazines that feed on an American fixation with lists," which is precisely what institutions say when they're trying to duck accountability.

There's a good reason for the American fixation with rankings -- if done correctly, they can help tell us what's working and what's not. Of course universities ought to be judged. The key is judging the right things.

All of the existing college rankings have the same aim -- to help overwhelmed parents and students sift through the thousands of colleges and universities in this country by giving them some yardstick for judging the "best" schools. Whether the guides actually do measure academic excellence -- as opposed to, say, academic reputation (not always the same thing) -- is debatable at best.

The publishers of these guides argue that they are providing a valuable consumer service. Parents who will shell out tens of thousands of dollars to put their teenagers through college need to know they are spending their money wisely.

How much more important, then, is it for taxpayers to know that their money -- in the form of billions of dollars of research grants and student aid -- is being put to good use? These are institutions, after all, that produce most of the country's cutting-edge scientific research and are therefore indirectly responsible for much of our national wealth and prosperity. They are the path to the American dream, the surest route for hard-working poor kids to achieve a better life in a changing economy. And they shape, in profound and subtle ways, students' ideas about American society and their place in it.

It seemed obvious to us that these heavily subsidized institutions ought to be graded on how well they perform in these roles, so we created the first annual Washington Monthly College Rankings. While other guides ask what colleges can do for students, we ask what colleges are doing for the country.

Iowa State beats Princeton

The first question we asked was, what does America need from its universities? From this starting point, we came up with three central criteria: Universities should be engines of social mobility, they should produce the academic minds and scientific research that advance knowledge and drive economic growth, and they should inculcate and encourage an ethic of service. We designed our evaluation system accordingly.

Given our very different way of measuring success, we suspected that the marquee schools routinely found at the top of U.S. News's list might not finish at the very top of ours -- but even we were surprised by what the data revealed. Only three schools in the 2006 U.S. News top 10 are among our highest-ranked: MIT, Stanford, and the University of Pennsylvania.

In addition, while the private colleges of the Ivy League dominate most rankings of the nation's best colleges, they didn't dominate ours -- only Cornell and the University of Pennsylvania made our top 10, and Princeton (tied with Harvard for the top slot on U.S. News's current list) was all the way down at #44, a few slots behind South Carolina State University.

Our list was also more heavily populated with first-rate state schools (the University of California system scored particularly well) than that of U.S. News, which has no public universities within its top ten. UCLA finished second in our overall ranking, UC-Berkeley third, Penn State University sixth, Texas A&M seventh, UC-San Diego eighth and the University of Michigan tenth. Each of our highest-rated schools are, by any reasonable national measure, academically serious schools. But they are not the super-elite -- the Harvards and Yales -- that normally dominate lists of the nation's "best" universities.

The schools that topped our list didn't necessarily do so for the reasons you might expect. MIT earned its number one ranking not because of its ground-breaking research (although that didn't hurt), but on the basis of its commitment to national service -- the school ranked #7 in that category, far better than most of its elite peers.

Similarly, UCLA, which finished second in our overall rankings, excelled in research and came in first in our social mobility rating because of its astoundingly high graduation rate given its large numbers of lower-income students. (Schools in the University of California's system were consistently high performers in this area: UCLA took top honors, with UC-Berkeley, UC-San Diego, UC-Davis, and UC-Riverside not far behind.)

At the same time, Princeton finished behind schools such as the University of Arizona and Iowa State -- schools with which it probably does not often consider itself to be in competition -- not just because of its comparatively low research numbers, which are perhaps to be expected given that the university doesn't have a medical school and considers its mission to be teaching, not research. What really did in Princeton were mediocre scores on national service and social mobility, categories in which it should have excelled.

Other priorities

Princeton's comparatively low ranking is evidence of something else indicated by our numbers. Schools that are similar in size, prestige, and endowment end up in very different places on The Washington Monthly College Rankings, largely because of decisions they have made about how to prioritize their resources or focus their energies. When it comes to social mobility, for instance, Harvard has about the lowest percentage of Pell Grant recipients in its student body of any school in the country.

By comparison, Columbia, whose institutional ambitions and prestige are similar to Harvard's, has twice as many lower-income students as its counterpart on the Charles River; Cornell has nearly three times the number. Public universities provide some equally interesting data: Both Indiana University and the University of Virginia are the most elite public institutions in states with populations of roughly similar wealth, yet the percentage of IU students who are Pell Grant recipients is nearly twice that of UVa.

On research, as well, the results are interesting. The big state schools finished somewhat higher than we had expected, and the super-elite schools (the Cal Techs and Harvards) fell somewhat lower. Even so, we were caught off-guard by some of the top finishers, including University of California's San Diego campus. UCSD is not normally considered among the elite UC campuses -- UCLA and UC-Berkeley have that distinction -- much less top-tier national schools. But it has quietly rounded up a formidable team of scholars. Nine Nobelists are on faculty at UCSD (Dartmouth, by comparison, has none), and the National Research Council recently ranked its Oceanography, Neurosciences, Physiology, and Bioengineering departments either first or second in the country. This concentration of talent translates into direct benefits for the surrounding community: Forty percent of the companies in San Diego's biotech corridor are spin-offs of research based at UCSD. These accomplishments landed UCSD in the sixth slot for research grants, and eighth on our overall rankings.

Perhaps the most striking data, however, is found in national service. Our measures here were simple: whether a school devotes a significant part of its federal work study funding to placing students in community service jobs (as the original work study law intended); the percentage of students enrolled in ROTC; and the percentage of graduates currently enrolled in the Peace Corps. All schools, large and small, are capable of excelling in these areas. In fact, we found that while some very small and nationally unknown schools have made an aggressive commitment to national service, most of the highest ranking U.S. News schools have not. The University of Portland, for example, finishes third in national service while Harvard lingers down at #75. Harvard obviously has far more resources than the University of Portland, and there's no question that it could match Portland's remarkable performance on service, if it chose to make a similar commitment to emphasizing that value among its students. But, at least by the criteria we set, it has not.

These service results haven't changed much since the first time we rated colleges on their commitment to national service. But there's one nice surprise: MIT leaped from near the bottom of the pack three years ago to near the top today.

We created a separate ranking for the nation's liberal arts colleges, and our results there confirmed these general trends. Some of the schools at the top of our list -- including Wellesley and Bryn Mawr -- are considered among the nation's most elite liberal arts colleges. But some schools we didn't expect -- Wofford College, #8 -- or had simply never heard of -- Presbyterian College, #13 -- crept into top slots. Though research rankings for both Presbyterian and Wofford were comparatively low, both schools produced extremely strong numbers for service, and performed well in the social mobility standings. And the traditional prestige schools didn't all benefit from the Washington Monthly ranking system. Williams, which U.S. News ranks as the top liberal arts school in the country, wound up at #14 on our list, one slot below Presbyterian, largely because of its weak service numbers.

Patriotic competition

A word on our criteria. This is the first Washington Monthly College Rankings. In future years, we would prefer to expand our criteria and develop an even more comprehensive measure of the qualities by which colleges and universities enrich our country. There's only one problem: Many of these data aren't available.

We would love, for example, to add a category measuring academic excellence. It's nearly impossible, however, to directly gauge the quality of education a student receives at a given school. Most ranking systems rely on measures of inputs -- such as the average SAT scores of the incoming class or the size of faculty salaries. But as Amy Graham and Nicholas Thompson noted four years ago, "[t]hat's like measuring the quality of a restaurant by calculating how much it paid for silverware and food: not completely useless, but pretty far from ideal."

There is one existing set of data that would do a great deal to answer that question: the National Survey of Student Engagement (NSSE). NSSE compiles such information as the average number of hours students at a particular school spend doing homework or meeting with professors outside of class -- measures which, studies show, are highly correlated with academic achievement. Unfortunately, the vast majority of colleges and universities refuse to grant NSSE permission to release their schools' scores to the public, and legislation to force them to do so, sponsored by Sen. Ted Kennedy (D-Mass.), has been stifled in Congress. Rather than use data that we believe doesn't accurately measure the quality of an undergraduate education, we chose not to include that category in this year's ranking. We hope to be able to add those measures in the future.

And while we're putting together our wish list, we have a few additional requests. We would prefer that the federal government require every school to report the percentage of Pell Grant recipients who actually graduate, but it doesn't. We would love it if schools kept a systematic count of which professions their graduates entered -- such as teaching -- but they don't. And we would be thrilled if the federal government tabulated how many of its employees came from which schools.

Still, we have tried to abide by the best principles of social science and used the best data available to generate the closest possible measures of the qualities we value. It pleased us to use metrics for success that were almost all within the means of even the most modest of our nation's universities. For that is more or less the point of this exercise. Succeeding on the Washington Monthly ranking (and succeeding at serving the country) is within the reach of most schools. Granted, most colleges are unlikely to catch up to Johns Hopkins on research overnight. But when it comes to service, Portland finishes well because it has made an institutional commitment to values that work. And in terms of social mobility, schools such as Alabama A&M and South Carolina State -- hardly considered academic powerhouses -- score very high because their graduation rates are well above what their Pell Grant numbers would have predicted.

The U.S. News rankings, and others like them, have had an impact. A growing body of reporting and scholarship shows that the criteria these guides use have sent administrators scurrying to increase the amount of money given by their alumni or the SAT scores of their incoming freshman in order to improve their score. Such measures have arguably very little impact on how well a school serves its student body, but as schools compete for students, every little thing -- including rising or dropping two spots on a list -- counts.

Imagine, then, what would happen if thousands of schools were suddenly motivated to try to boost their scores on The Washington Monthly College Rankings. They'd start enrolling greater numbers of low-income students and putting great effort into ensuring that these students graduate. They'd encourage more of their students to join the Peace Corps or the military. They'd intensify their focus on producing more Ph.D. graduates in science and engineering. And as a result, we all would benefit from a wealthier, freer, more vibrant, and democratic country.

Liked this story? Get top stories in your inbox each week from AlterNet! Sign up now »