Leveraging Dollars with Data

A data warehousing project coming online in late 2012 will help Virginia lawmakers forge education policy based upon hard data instead of anecdote and ideology.

By  James A. Bacon

Tod Massa is big man with big ideas. His goal is to build a “Longitudinal Data System” (LDS) that will track the progress of Virginians from pre- Kindergarten through high school, into college and then through their working lives. With a bear-like frame and a broad, bearded face, he speaks passionately about the potential of the $17.5 million project to “fundamentally transform” education and workforce-development policy in the Old Dominion.

When crafting policy, law makers rely upon anecdote and hunch, ideology and politics, and patchy, often-inconclusive research. The state has lots of data but it resides in different silos so it’s of limited use. The Department of Education (DOE) maintains data on K-12 students such as enrollment, graduation rates and test scores. The State Council for Higher Education in Virginia (SCHEV) keeps track of college enrollments, degrees earned and financial aid. The Virginia Employment Commission has access to employment and wage/salary data. But none of the silos connect.

The DOE can’t tell, for instance, how many graduates of a particular high school went on to earn a college degree, much less how much they earned when they entered the workforce. By December 2012, however, the LDS will link the separate databases, allowing analysts to get definitive answers to that type of question. If it can dispel the uncertainty that fogs so much debate over education and workforce development, a modest investment in data warehousing will pay for itself hundreds of times over.

“It’s an exciting time in higher education and state government,” says Massa. Virginia, like other states, has been engaging too long in “drive-by school improvements,” he says, spending vast sums of money to little effect. Soon it will be possible to answer many questions not with a survey, not with a sample, but with hard data on what is. Pet theories from all across the ideological spectrum will bite the dust and a more effective educational system will emerge.

As director of policy research and data warehousing for SCHEV, Massa represents one of three key agencies that are collaborating on the project. Virginia, he says, is one of 20 states participating in a federal initiative funded under the American Recovery and Reinvestment Act (widely known as the “stimulus” bill). What will be distinctive about Virginia’s linked databases is a mechanism for “de-identifying” individual identities in the database. Names, social security numbers and other tags will be stripped out and replaced with numerical codes. In the Old Dominion, he insists, it will be “close to impossible” for anyone’s privacy to be violated.

A top priority in educational policy right now is to measure how effectively teachers, principals and schools educate students. Virginia’s LDS will allow analysts to track, among many other things, how well a teacher’s students perform in standardized test scores. That kind of study has been accomplished on a small scale before, but Virginia will be able to roll it out statewide. Such data can be used, with appropriate adjustments for those who teach more challenging students, as the basis for rewarding good teachers and weeding out bad ones.

The list of questions is endless. What’s the payoff from investing in universal pre-K? Are college grads earning enough to pay back their student loans? Are certain majors valued more highly in the workplace?

Peter Blake, interim SCHEV director, gives another hypothetical example: Should the state channel its financial support for higher ed into undergraduate programs or should it support graduate schools, even though they cater extensively to out-of-state residents? One way to approach that question is to find out how many graduate students at Virginia colleges and universities end up getting employed in Virginia and how much money they earn. It might be easier to justify state support, he says, if it can be shown that grad schools contribute to the build-up of Virginia’s human capital.

Massa envisions researchers rummaging through the data and looking for patterns and outliers. For example, only 30% to 50% of the students attending Virginia community colleges complete a degree (with an in-state institution) within 10 years. But the figure for Richard Bland College, a two-year college in Petersburg, is 77%. Why is that? Is there something unique about the student population, or is the college doing something different? Perhaps other community colleges can learn from its example.

What really gets Massa stoked is the idea of letting scholars and members of the public use the data to generate their own findings. Qualified researchers will be given access to all the data under contract and, he hopes, a broad selection of the data will be made accessible to anyone with a Web browser.  He sees citizens being able to build their own data dashboards, create reports and publicize their findings. “I want the data shared,” he declares. Ninety percent of the citizen analysis may be worthless – but 10% might be spun into public-policy gold.

There are currently no comments highlighted.

11 responses to “Leveraging Dollars with Data

  1. the basic problem is that if a child does not have a good foundation for reading, writing and math by the time they leave K-6 – the die is cast and it would be a shame to capture all this other “after the fact” data.

    we need to know WHY kids don’t achieve in middle and high school and it’s NOT due to a teacher when the kid moves into middle and high school as a functional illiterate.

    I can’t tell if this program is connected to UVA PALS but PALS is where it is at for K-6 literacy:

    http://pals.virginia.edu/rd-background.html

    it cannot be overemphasized – the game is over if the kid hits middle school and is not literate….

    all the programs and money in the world is not going to help a lot of those kids because without basic literacy – you cannot learn -…… anything that is taught… those kids basically stumble through those courses…and teachers at the middle and high school levels are presented with impossible situations when they get kids who are not proficient in the basics.

  2. Once again, progress during the McDonnell Administration.

    This is exactly what is needed. If some middle school is consistently producing graduates who cannot get through high school or who can find employment, then it is time to take a long hard look at that middle school.

    If some 5th grade English teacher consistently passes on students who can’t pass proficiency tests for English then it is time to take a long, hard look at that teacher.

    Unfortunately, I remain a sceptic of whether this will happen. Many powerful vested interests will find the glare of sunlight upon their formerly concealed ineptitude.

    For example, I’d love to see the real statistics from the University of Virginia. What are the average SAT scores of students accepted into UVA by region of the state? How many “international” students have been accepted? How many students with engineering degrees stay in the state?

    Connecting the dots between education and employment would be a huge accomplishment.

    I hope this data warehouse gets built. I hope the anonymous data is made available to everybody for study.

  3. As much as McDonnell administration deserves praise for making Virginia one of the 20 participating states, we also must credit the Obama administration for coming up with the funding for this initiative through (it pains me to say this) the much-maligned stimulus plan. If Massa has his way and the Virginia LDS becomes the template for national system, more kudos could be coming McDonnell’s way. The real takeaway is that this is initiative is a rare instance of government transcending partisan and ideological bickering.

  4. Well hell, as long as we are going cradle to grave, why not add in FICO scores or even a full financial disclosure with a current resume? Maybe have the medical group add in all the odd ailments just so there is full transparency in following little Johnny to his final countdown. Oh but then we would have to raise taxes to pay for the super-duper Cray clone to process everything.

    That’s the bad thing about data. There is never enough nosiness.

  5. “This is exactly what is needed. If some middle school is consistently producing graduates who cannot get through high school or who can find employment, then it is time to take a long hard look at that middle school.

    If some 5th grade English teacher consistently passes on students who can’t pass proficiency tests for English then it is time to take a long, hard look at that teacher.”

    you’re going to find that the teacher does what they must to keep their job.

    if you fail kids that should fail – and it in turn causes problems with NCLB – you’re gonna get whacked …AND you’re going to get “tagged” with not being a “team player”.

    this is not a teacher problem – it’s an institutional problem and it’s the reason why NCLB is so reviled by school administrators.

    “Unfortunately, I remain a sceptic of whether this will happen. Many powerful vested interests will find the glare of sunlight upon their formerly concealed ineptitude.”

    NCLB is an excellent transparency but look at what happens. Not accountability but advocacy to kill it.. or cheat on it.

    “For example, I’d love to see the real statistics from the University of Virginia. What are the average SAT scores of students accepted into UVA by region of the state? How many “international” students have been accepted? How many students with engineering degrees stay in the state?”

    when almost 1/2 of your high school graduates cannot pass the armed forces aptitude test – why are you worrying about college and SAT?

    those kids who can’t get into the armed forces – also cannot get a job… will not pay Federal Taxes and will need entitlements including health care.

    there is no escape from this – unless we want to throw up our hands and revert back to a 3rd world country status.

    “Connecting the dots between education and employment would be a huge accomplishment.

    I hope this data warehouse gets built. I hope the anonymous data is made available to everybody for study.”

    here’s the first thing that most folks don’t know and that is that the Feds fund about 1K a year in education but almost 100% of it goes to K-6 remedial education for kids who cannot read and write.

    Imagine what happens when we kill the Fed Dept of Education and NCLB.

    you might ask – why localities don’t fund education for these at-risk kids and why the Feds have to.

    when you can successfully answer than question.. please share it.

  6. the only reasonable rationale for taxpayer funding of public education is for an employable workforce that results in people being able to pay taxes and not needing entitlements.

    we have one seriously off track with K-12 education being primarily an exercise in college resume-building for some kids – while we don’t give a rats behind for the other kids – even though the “good” kids will end up with all the costs of funding government …entitlements ..and incarceration.

    it’s like we can’t help from being stupid about the whole thing.

  7. Hi. Jim wrote a good article, but I must not have explained something well enough. We are not building a data warehouse. Instead, we are building a system that allows existing data warehouses to be connected to match data as needed. While this is may appear to be no more than a technical distinction, for a number of reasons, including Virginia law, it is an important distinction.

    All the points raised in the preceding comments are familiar to me. This project started way back in 2009 with development of the proposal. From even before that, when the P16 Council proposed something similar as far back as 2006, we have wrestled with these questions and many others.

    This system won’t solve these problems. I do think it will change the discussions about policy and lead to significant improvements over time. I hope people like you will continue to raise questions, more importantly, as the system and the data from the system become available, you will help us study the data and find more ways to make a difference.

    -Tod Massa

  8. Thanks for the correction, Tod. I have changed the article to refer longitudinal data systems and linked databases.

  9. Anything that can help move public discussion of education beyond emotion and slogans is a positive step forward. Too often, the discussion never goes beyond “The schools are too expensive” and “It’s for the children.” What works? What does not? How do we know? Which schools are making continuous improvement? What are they doing? Can it be replicated?

  10. we’ve gotten on a very wrong track on the reasons why education is failing. It has virtually nothing to do with “bad” teachers – not when the whole school is showing lousy numbers – which is more often than not the case.

    we spend time looking for easy scapegoats.

    Most teachers are milk-toast when it comes to their job. They invariably do what they are told to do even if they are union but certainly in right-to-work-states (like Va).

    teachers are like most employees. They are going to do what it takes to be like other teachers who keep their jobs.

    most teachers stay between the lines and try to keep their jobs.

    they are the easiest to blame but the least at fault.

Leave a Reply