The New College Scorecard: NPR Does Some Math

By Anya Kamenetz

Published September 21, 2015 at 4:02 AM CDT

The Obama Administration's long awaited, and slightly-different-than-planned College Scorecard is open ... for interpretation.

The new tool combines data from the Treasury and IRS with Department of Education records on more than 7,000 colleges and universities, going back 18 years. Anyone can access the data that shows how particular colleges are doing at enabling students to pay back loans and pay their bills.

Anyone, including our colleagues over at NPR's Planet Money team.

Planet Money used the data to create its own rankings of four-year programs, something the Department of Education carefully avoided doing.

They reached out to three different higher ed experts — Peter Cappelli, a professor at UPenn's Wharton Business School, Anthony Carnevale of the Georgetown Center on Education and the Workforce, and Amy Laitinen at the New America Foundation. They asked these experts how they'd weight the various pieces of information available, and came up with three different top-50 lists.

Together, these rankings show just how complex this process is.

For example, if you focus just on income later in life, you'll see Babson College, a private school in Wellesley, Mass., that exclusively offers business and entrepreneurship programs, in the No. 5 spot. Engineering schools — Harvey Mudd in California and Georgia Institute of Technology — make strong showings as well.

If you include default rates on student loans, Duke fares better than Harvard. But that measure may be misleading, because far more students borrow at Duke. The 3 percent of students who take out federal loans at Harvard may have other severe financial challenges.

Then again, if you are interested in how colleges serve low-income and first-generation students, suddenly the top spots are full of public institutions, especially in California.

Two years ago, when the White House announced this project, it sounded more like the administration was developing rankings like these. The project was described as a Consumer Reports-style ratings system for postsecondary institutions. There was even speculation that the system might have teeth in it — by being linked to eligibility for federal aid, for example.

But they ran into all kinds of objections from college presidents and organizations who said that any metrics, no matter how complex, wouldn't give you a way to compare the mission and special appeal of say, Emmanuel College, a Pentecostal-affiliated private school with 700 students in Georgia, with the University of California, Davis, which has 26,000 undergraduates.

So the government's tool as released avoids rating or ranking colleges.

In some ways, the differences among the lists Planet Money came up with prove the point that any particular ranking is somewhat arbitrary. The real question is whether all this information will help parents and students make better decisions.

Lisa Gelobter, who took the lead in designing the government's tool, says that they'll continue to do user-testing with high school and adult prospective students, and collect detailed information through users of the site to tweak and improve it. "To succeed is to get it into the hands of students," she said. "We want to do whatever we can to ensure that."

For more details on the methodology behind these rankings, go to NPR.org/money.