Outcomes Roundup: 4 New Tools to Measure ROI of a Degree
Outcomes are increasingly a part of the conversation that students and their families are having when they make decisions about higher education. Over the last five years, a number of new tools have appeared to help them evaluate the return on investment of a degree.
Each tool takes a slightly different approach to organizing information, and varying results can be found across tools. With this year’s rerelease of the Education Department’s College Scorecard, more and more iterations of rankings will appear, face questions, yet ultimately become a part of a student’s decision process. The question is: Will your program take ownership of gathering and reporting on rankings, or will you leave the story in someone else’s hands?
Here’s a quick synopsis of new or updated tools so far in 2015 and the main data points they use to formulate their rankings:
PayScale’s College ROI Report
Released
First edition June 2010, most recent edition March 2015
Goal
From PayScale: “We've provided the ability to view the best value colleges for various majors and career paths as well as evaluate ROI at a school overall. You can see which colleges are providing the best monetary return for their alumni via low cost of attendance, high earning potential or a combination of the two.”
Where the data comes from
Data comes from employees who completed PayScale’s employee survey. Notes on the methodology can be found here.
How schools can use it
PayScale provides embeddable badges that “best value” schools can place on their websites.
The Education Department's College Scorecard
Released
February 2013, updated September 2015
Goal: From the White House: “[College Scorecard was] redesigned with direct input from students, families, and their advisers to provide the clearest, most accessible, and most reliable national data on college cost, graduation, debt, and post-college earnings. This new College Scorecard can empower Americans to rate colleges based on what matters most to them; to highlight colleges that are serving students of all backgrounds well; and to focus on making a quality, affordable education within reach.
“The old way of assessing college choices relied on static ratings lists compiled by someone who was deciding what value to place on different factors. The new way of assessing college choices, with the help of technology and open data, makes it possible for anyone – a student, a school, a policymaker, or a researcher – to decide what factors to evaluate.”
Where the data comes from
College Scorecard includes 20 years of data for 7,000-plus colleges and universities from sources including IPEDS, NSLDS and Department of the Treasury. Information on data documentation can be found here.
How schools can use it
Data is made available for download through an open application programming interface (API) to allow for custom analysis.
The Brookings Institution’s “Value-Added” Approach
Released
April 2015, updated October 2015
Goal
From Brookings: “Value-added measures attempt to isolate the contribution of the college to student outcomes, as distinct from what one might predict based on student characteristics or the level of degree offered. It is not a measure of return on investment, but rather a way to compare colleges on a more equal footing, by adjusting for the relative advantages or disadvantages faced by diverse students pursuing different levels of study across different local economies … Colleges have very different missions and serve diverse populations with varying levels of academic preparation. Value-added measures attempt to account for these differences in order to evaluate colleges on an even playing field.”
Where the data comes from
Data on alumni earnings, student loan repayment rates within three years after enrollment and occupational earning power was drawn from PayScale, LinkedIn, Bureau of Labor Statistics and College Scorecard. More information about Brookings’ value-added methodology can be found here.
How schools can use it
Limited to sortable table on Brookings’ website that covers 3,173 institutions (includes two-year, four-year and vocational schools).
The Economist’s College Rankings
Released
October 2015
Goal
From The Economist: “The Economist’s first-ever college rankings are based on a simple, if debatable, premise: the economic value of a university is equal to the gap between how much money its graduates earn, and how much they might have made had they studied elsewhere. Thanks to the scorecard, the first number is easily accessible. The second, however, can only be estimated. To calculate this figure, we ran the scorecard’s earnings data through a multiple regression analysis, a common method of measuring the relationships between variables. We wanted to know how a wide range of factors would affect the median earnings in 2011 of a college’s graduates.”
Where the data comes from
Data gathered for four-year, nonvocational American colleges comes directly from the scorecard, including average SAT score, sex ratio, race breakdown, college size, institution classification and the mix of subjects students chose to study. Additional sources were consulted to determine a school’s religious affiliation, the wealth of its state, typical wages in its city, if it has a ranked undergraduate business school, percentage of students who receive federal Pell Grants, and a custom “Marx and Marley” index to evaluate students likely to pursue less lucrative careers. Additional information on data sources and The Economist’s multiple regression analysis can be found here.
How schools can use it
Limited to sortable table on The Economist’s website that covers 1,275 institutions.
While a number of articles offer critiques regarding these ranking tools, the fact of the matter is that students and their families want to know what outcomes they can expect from their educational investment. Tools like Seelio can help you demonstrate student outcomes. How are you taking on the challenge of demonstrating student outcomes? Start a conversation with us on Twitter @KeypathEDU.