Measuring Educational Value Added

Lexington -- national center of value-added education.
Lexington — national center of value-added education.

by James A. Bacon

What are the top colleges and universities in Virginia? We know the usual roster, based upon the annual survey by U.S. News & World Report: The University of Virginia, College of William and Mary, and Virginia Tech. Essentially, U.S. News measures the prestige of an institution. But how well do colleges and universities actually actually prepare students to earn a living? That’s a very different question, and it’s one that that the Brookings Institution has set out to answer with a very different kind of study in “Beyond College Rankings: A Value-Added Approach to Assessing Two- and Four-Year Schools.”

A university can have immense prestige based upon factors such as the star power of its faculty, the size of its endowment or the average SAT scores of its student body. But if the faculty stars delegate much of their teaching to graduate students and the endowment underwrites the building of magnificent edifices, and if students spend more time partying than studying, prestige may not translate into effective learning. Conversely, an institution whose faculty members excel at teaching rather than, say, publishing books and winning research grants might actually do a better job of preparing their students for the world beyond.

eva
In the Brookings rankings, 100 is the highest score.

Based upon Brooking’s methodology, Washington & Lee University provides the most educational value added in Virginia, followed closely by Virginia Military Institute. Virginia Tech surpasses the University of Virginia, and the College of William and Mary looks decidedly mediocre. The performance of Virginia’s community colleges looks especially dismal. It’s a very different profile than the U.S. News rankings that allow Virginians to proclaim that the Old Dominion provides the best undergraduate education of any state in the country.

Brookings compares actual economic metrics such as mid-career earnings, occupational earnings potential and repayment of federal student loans to the “expected” level based upon race, ethnicity, family income and academic preparation. The greater the gap between actual and expected, the greater the value added. It is important to note that this methodology does not capture all value from a college education, such as a person’s intellectual, artistic or spiritual development or a person’s preparation for civic or political participation. However, insofar as the primary justification most people give for attending college is to prepare for a career and increase their earnings potential, Brookings arguably captures the most important data.

Due to its immense prestige, Harvard University attracts some of the brightest students from across the country. Many of those students would be successful in life no matter where they attended college, or even if they dropped out. It’s no surprise that Harvard graduates earn a lot of money. As it turns out, Harvard still performs better than most institutions in adding value, giving a bigger edge to already advantaged students. But in terms of creating economic value added, Washington & Lee in Lexington, Va., out-performs Harvard, while VMI, also in Lexington, almost equals it.

A fascinating aside: One could argue that tiny Lexington is a national center of excellence for economic value-added education. One small Virginia town is home to two of the highest ranking institutions in the Brookings list.

Bacon’s bottom line: Brookings’ calculus is terribly complex and, truth be told, I have not had time this morning to do any more than skim the surface. I am in no position to evaluate the methodology, and I’m sure that many institutions (especially those who fare below expectations) will take exception to it. But I will say this: Brookings is asking the right questions. Educational institutions should be judged not on their prestige but upon their ability to deliver tangible value to students. If there are flaws in the Brookings approach, let’s fix them and keep moving.


Share this article



ADVERTISEMENT

(comments below)



ADVERTISEMENT

(comments below)


Comments

  1. Steve Haner Avatar
    Steve Haner

    The math is extremely complex. I haven’t read the study closely, either, but on first blush it seems to be measuring how well the schools do vs. expectations. In which case any school with a positive number is doing better than expectations. On several of the measures the national average of “value add” was in single digits or even a negative number, according to the report. I see no negative numbers on your selected list, and only a couple of single digits.

    While VMI and W&L seem to stand close to the top, it looks to me like most of the schools you listed can say: Get a degree from us and there is significant value added to your earning potential. Given the incoming statistics of students at W&M and UVA they start with high expectations, yet the graduates exceed those expectations. They are not mediocre at all.

    Could I be reading this wrong? Yes, it is a very dense report.

  2. Peter Galuszka Avatar
    Peter Galuszka

    I don’t have time to wade through the report, but my first impression is that the hypothesis is utter crock. W&L, a nice, tight conservative school in a nice little town, produces a bunch of entitled pre-laws, who go to good law schools and then work to screw the rest of us, getting well paid in the bargain. What’s more, I never knew W&L had a strong science and technology research program. Whatever happened to the beloved STEM?
    The problem is that this kind of thinking reduces teaching and learning about humanity and science to a trade school. They have their place and are, at least, honest about it.

    1. virginiagal2 Avatar
      virginiagal2

      Washington and Lee has a reputation as a first rate liberal arts school, and has been rated as such for some years on a variety of lists.

      A quick perusal of their Wikipedia page shows they were ranked 14th in the country for liberal arts schools by USNWR, and the law school was 42nd. The section on their academics and reputation shows a fairly long list of other top rankings and positive information.

      I don’t know about research programs, but it has a wide selection of STEM majors.

      The people I know who went to W&L aren’t any more or less conservative than the people I know who went to other first rank Virginia universities. They also weren’t any more (or less) entitled and not any more (or less) likely to go to law school.

      What’s the point of stereotyping a school? This just seems mean, and very unfair.

    2. LifeOnTheFallLine Avatar
      LifeOnTheFallLine

      The analysis resulted in the wrong type of school from the wrong place finishing first so the metrics must be wrong…

  3. larryg Avatar

    I’m not sure how you’d do value added but I don’t think this is it.

    for my money maybe an ROI ? and throw out anything beyond the basic BS and BA – and oh, by the way – not the average salary – the median and a 2 sigma histogram.

    I heard a tale of a lady principle in a state serious about education who was excepted to get a PHD and she did – and now she’s in debt 80K … and the word is – it make take decades to pay it back.

    anyone else know if principles in Va are expected to attain PHD?

    I notice at School Board meetings up my way all the administrators are addressed as Dr. This and Dr. That.

  4. Cville Resident Avatar
    Cville Resident

    I’d take it with a grain of salt….as Mr. Haner points out, it’s inherent bias is against elite schools which are more likely to admit students who are “supposed to be” successful.

    But one point is really not debatable no matter what metric you examine. I talked to a former SCHEV staffer a few months ago. She told me that the single biggest higher ed money waster in Virginia is VCU. If anyone was serious about higher ed in the Commonwealth, VCU would have been shut down yesterday, and MCV would be a standalone institution. Its student outcomes are poor. Its prestige is poor. Its research funding is poor. Look at ranking after ranking and VCU is always poorly ranked except for fine arts and social work.

    Of course, everyone in Richmond bottles all this up b/c they’ve deluded themselves into thinking VCU somehow lead to a “Richmond Renaissance.” Nothing actually backs up that claim except for real estate development….which isn’t exactly a university’s function.

    1. LifeOnTheFallLine Avatar
      LifeOnTheFallLine

      “VCU is terrible except for these programs that are among the best in the nation.”

      A+ trolling, would LOL again.

  5. larryg Avatar

    One of the others things that may better inform us is this: We pretty much know what’s happened job-wise for folks who did not go to college. Manufacturing is going away and taken over by robots, etc. stubborn unemployment for those without college as well as much lower lifetime earnings, etc..

    but what kinds of work – are college graduates doing – that still exists and still pays well and has that world (college-degreed) of employment changed or stayed the same – and perhaps as important – are some jobs that require college – themselves started to be affected by computerization and big data?

    I think the emphasis on STEM – pure STEM has gone sideways myself.

    what employers are looking for is people who are capable of exercising critical thinking skills in carrying out their duties – and one shorthand way of getting some assurance that a new employee is capable of that – is if they took some hard science or engineering courses that usually cannot be passed if you don’t have or don’t use critical thinking skills – in dealing with today’s technology and enterprise concepts. As we already know – some college degrees don’t necessarily lead to a sure job – other than Starbucks.

    So I’d be interested in knowing for a given college – how many folks graduate with those kinds of degree that end up not being very valuable in post-school world.

    Another metric – might be how may jobs in a field are available vs how many college grads are competing for them.

    in other words there are jobs available but perhaps only 1 per 10 college grads competing for them.

    why is that important?

    2 reasons

    1. – in addition to means-testing financial aid, we should ROI-test it. Giving thousands of dollars of financial aid for a degree in a field that is already overrun with applicants is simply not a legitimate use of govt-guaranteed loans and our college loan policy should be aimed at ROI for taxpayers not self-actuation for Johnny.

    2. – for kids not on a 4yr college track – the k-12 should be pointing these kids to jobs in the private sector that do exist – and do require – critical thinking skills even at a 2yr college level – not just any courses at a 2yr college.

    Finally – I think in terms of the fiscal stress on colleges and taxpayers and the colleges tendencies to try to build and maintain academic fiefdoms – funded by taxpayers – in fields of study that while important – don’t return on dollar – I would assert those areas are not legitimate areas of as much or full funding from taxpayers.

    Let the schools that are good at liberal arts do their thing – but let them do their thing with the folks that want their kids to get that kind of education even if it does not lead to a job.

    the fundamental purpose of taxpayer dollars for ANY public education is – ROI – not what Johnny wants to do – to “follow his dream” – at least not the way we’re doing this now where if Johnny comes from a family of some – even if modest means – he can do that on the taxpayer dime – but economically-disadvantaged kids do not have the same bite at that apple – and you’d certainly not want to be providing taxpayer dollars to THEM to “follow their dreams” – we need more employed people and less entitlement takers and that’s where our limited Higher Ed dollars should be going.

    in m “libtard” view of course which is way more fiscally conservative than some of the folks who throw the “libtard” label at others in this forum!

    we need to get our collective heads screwed on straight about the PURPOSE of education – it’s aint just to make Johnny feel good and his parents proud.

    1. virginiagal2 Avatar
      virginiagal2

      Okay, let’s look at the varied purposes of higher (college) education.

      First, there’s the filter effect. There is a very definite perception that, if you got in to an elite university, there’s something special about you, regardless of how you did once you got there. That perception opens doors.

      Second, there are the connections you form, both primary (the people you go to school with), and secondary (their connections in turn.)

      Especially for elite universities, I believe much of the subsequent career advantage comes from factors one and two above.

      Third, there’s the opportunity to learn more about the world and become a better person, by thinking more widely, and the chance to connect to a subject that opens your mind to new possibilities. I’ve personally definitely seen that – the downside being that, after 4 years at UVA, there were about four classes that really had a truly memorable effect on me, out of about 40-45 classes all told.

      Fourth, there are the soft skills that are supposed to help prepare you for doing a job – research, writing, etc. This is said with the caveat that the writing and research needed for business generally doesn’t much resemble what you do for research papers in college.

      Fifth, there are work-related skills that you learn, that directly prepare you for doing a job – programming, engineering, accounting, marketing, etc.

      Sixth, a lot of kids are going to college so they can go to graduate or professional school (med school, law school, MBA, dentistry, veterinary medicine, etc.)

      Currently, only about 15% of college students are studying a liberal arts curriculum. The rest are in some type of directly career-oriented program (business, engineering, education, health-care, etc.)

      If the emphasis is on providing people with better jobs, I’m not sure that blindly directing people into college is going to have the expected payoffs. It seems like what we want is the fifth purpose – career education – but we’re going about it in a way to maximize the third (personal growth and citizenship.)

      I’m not so sure kids are going into 100K of debt so that they have a greater appreciation of Melville.

      Maybe we, as a society, need to re-examine what we want from higher ed, and see if there are better ways to structure how we provide career training and personal growth.

  6. larryg Avatar

    we cannot buy all of these things with tax dollars. And people who do go – ARE RESPONSIBLE for living within their means. Jut because loans are available does not mean you should go into debt over your eyeballs.

    I do not think it should be “what we want”. I think it should be what we need economically as a country – verses what a parent or child might want -beyond that need – and should be paying for it.

    we’ve developed an attitude with education and health insurance – that the dollars should cover “everything” and we simply cannot afford that – and beyond that it fosters an irresponsible individual ethic – that anything we want – we deserve it being provided to us …

    we castigate the poor for their perceived expectation of entitlements – but I would argue – we have that same mentality for college… and it’s destroying our ability to afford it – as taxpayers and as individuals.

    we’re no longer being responsible for consuming what we can afford and moving on to being productive …both individually and collectively.

    the first thing is employment – as much as we can accomplish that – but certainly not to let all other wants essentially make that merely “one” option.

    If we don’t get our act together on this – we’re going to continue to sink against other countries – and become a “has been” economic power.

  7. larryg Avatar

    One of our problems is our attitude towards 4yr college verses other schooling and training.

    we treat 4-yr college as the preferred Creme De La Creme as if it was the only option that really matters – and that it should offer soup-to-nuts for everyone who goes – regardless of cost or relevance in the employment world.

    i.e. a BA from Podunk U – is worth “more” than an occupational certificate from a Community College.

    I would argue that having that guy who went to community college EMPLOYED is every bit as useful and needed as having that 4year grad waiting tables until they find their “place” in this world and can start paying back their loans.

    we’ve evolved on this to a rancid state where “education” is not really valued unless it fulfills folks own kids wants and desires and hang the cost if it is coming from subsidies… that actually compete against buying a K12 grad an occupational certificate from a Community College.

    Having a kid who did “wonderful” at AP in K-12 who then ends up with a BA and works as an unpaid or lower paid intern as a de-facto ‘gopher” with an “Assistant” title – is not … “wonderful”.. it’s sad..and it squanders our educational resources.. in my view.

    our ethic should be – get a job .. and on your way – get the education you need to do that job.

  8. larryg Avatar

    WOW!
    ” What new graduates expect to earn in their first job is pretty different from what grads of 2014 and 2013 have actually been making, according to a survey released Tuesday by the consulting firm Accenture.

    The survey found that while just 15 percent of the class of 2015 expects to make $25,000 or less after graduation, a stunning 41 percent of the classes of 2014 and 2013 is earning in that range.

    Nearly half of the class of 2015 said they expect to make $40,000 or more annually in their first job, according to the survey. Only a quarter of the classes of 2014 and 2013 earns that much.”

    http://goo.gl/YTx4jg

  9. Townie76 Avatar
    Townie76

    As a native of Lexington and a graduate of one of the two schools highlighted, I find some of the comments denigrating Washington and Lee to be unfair (btw I went to VMI). Yes Washington and Lee was a very conservative all male white bread school, it ain’t now. It may be conservative but over 50% of its students are female and it attracts a very diverse student body.
    Those question whether Washington and Lee and VMI should be on the list ahead to THE UNIVERSITY or whatever missed a key point–at Washington and Lee and VMI you do not have Graduate Assistant teaching–you get the actual professor and they teach in addition to doing research and publishing.
    Washington and Lee a Liberal Arts school has always had a strong science program and VMI likewise has a very strong science and engineering program as well as a strong Liberal Arts program. Both schools seek to produce students who are prepared for the present by being well rounded critical thinkers and are adaptable and flexible enough to change with the future.
    Thus endth my rant.

  10. […] other day, Jim Bacon published a piece on the Brookings study, Beyond College Rankings: A Value-Added Approach to Assessing Two- and […]

  11. Richard Avatar
    Richard

    W&L is a great school by many measures. I am a UVA grad but went to law school there. Numerous relatives have gone there including most recently a couple of smart nieces. My impression is that you can get a good STEM education or other liberal arts education there. It’s a small school with dedicated professors, and they know most or all of the students in their departments. This is a tremendous advantage for many students – both those who are stars and who would do well in any educational environment, and those who need some support. Think about the hordes of “first years” taking intro to chemistry at UVA from assistant professors and graduate assistants, and compare those to freshman/women at W&L who are in a relatively small class that might be taught by the head of the department.

Leave a Reply