A College Ranking to Virginia’s Liking

There are plenty of people in the college rating game these games, from the venerable US News & World-Report to Forbes magazine to the Wall Street Journal. Results vary depending on the criteria selected and the weight assigned to those criteria, both of which entail decisions and value judgments subject to human bias. But what if Artificial Intelligence was used to compile the rankings?

That’s what MetaMetrics, a Durham, N.C.-based company specializing in educational metrics, has tried to do. MetaMetrics research engineer Steve Lattanzio explains:

Was it possible to have a computer algorithm take in a bunch of raw data and, through a sufficiently black-box approach, remove decision points that allow ratings to become subjective? … Could an artificial intelligence discover a latent dimension hidden behind all the noise that was driving data points such as SAT scores, admission rates, earnings, loan repayment rates, and a thousand other things, instead of combining just a few of them in a subjective fashion?

The company drew upon the College Scoreboard, an exhaustive U.S. Department of Education database on colleges, students, and student loans. Lattanzio continues:

We use neural networks to perform “representational learning” through the use of what is called a stacked autoencoder. I’ll skip over the technical details, but the concept behind representational learning is to take a bunch of information that is represented in a lot of variables, or dimensions, and represent as much of the original information as possible with a lot fewer dimensions. In a stacked neural network autoencoder, data entering into the network is squashed down into fewer and fewer dimensions on one side and squeezed through a bottleneck. On the other side of the network, that squashed information is unpacked in an attempt to reconstruct the original data. …  the AI isn’t figuring out which subset of variables it wants to keep and which it wants to discard; it is figuring out how to express as much of the original data as possible in brand new meta-variables that it is concocting by combining the original data in creative ways. …

It turns out that we were able to compress all of the information down to just two dimensions, and the significance of those two dimensions was immediately clear.

One dimension has encoded a latent dimension that is related to things such as the size of the school and whether it is public or private (in fact, the algorithm decided there should be a rift mostly separating larger public institutions from smaller schools). The other dimension is a strong candidate for overall quality of a school and is correlated with all of the standard indicators of quality. It seems as if the algorithm learned that for higher education, if you must break it down into two things, [the data] is best broken down into two dimensions that can loosely be described as quantity and quality.

Got that? Good. So, here are the results for the top 20 colleges:

  1. Duke University
  2. Stanford University
  3. Vanderbilt University
  4. Cornell University
  5. Brown University
  6. Emory University
  7. University of Virginia
  8. University of Chicago
  9. Boston College
  10. University of Notre Dame
  11. College of William & Mary
  12. University of Southern California
  13. Wesleyan University
  14. Yale University
  15. Massachusetts of Technology
  16. Northwestern University
  17. Bucknell University
  18. University of Pennsylvania
  19. Santa Clara University
  20. Carnegie Mellon University

What? No Harvard or Princeton? Correct. The AI does not take into account intangible factors such as prestige. By the AI’s reckoning, it appears, those institutions are over-rated.

Virginia higher-ed officials looking for bragging rights can surely find them with this methodology — at least if they don’t dig too deep. UVa ranks 7th in the country and W&M ranks 11th. They are two of only three public universities on the list. The University of Richmond, described as a “hidden ivy,” logged in at 32nd, while Washington & Lee University scored 63. As comedian Larry David might say, that’s pretty, pretty impressive.

Virginia’s non-elite public universities scored fair to middling, according to the AI’s way of thinking. Out of 1,313 institutions nationally:

James Madison University — 146
Virginia Tech — 157
Virginia Military Institute — 199
George Mason University — 316
Radford University — 482
Longwood University — 495
Virginia Commonwealth University — 504
Old Dominion University — 951
Norfolk State University — 1,164
Virginia State University — 1,213

I could find no mention of Mary Washington University or the University of Virginia-Wise.

MetaMetrics provides plenty of caveats, which you can read here. The ranking “is not perfect and the rankings should not be viewed as infallible,” writes Lattanzio. “But when viewed among other college rankings, its validity is undeniable. It’s not merely a measure of prestige, and it addresses most of the concerns of critics of college rankings, while undoubtedly raising some new ones.”

I do fine one thing very curious. The company is located in Durham, N.C., home of Duke University. Four of the company’s top 11 senior executives have Duke affiliations — as does Lattanzio himself. Who ranks as the No. 1 university in the country? Duke, of course. Pure coincidence? Let’s just say, when Duke plays the University of North Carolina in basketball, you can probably find the AI in the stands rooting for the Blue Devils.

(Hat tip: Mary Helen Willett)