Bacon's Rebellion

Democracy Thrives in Sunlight

Another Useless Educational Metric

—

by

James A. Bacon
in

local_pass_vs_SOL2by James A. Bacon

We have documented in previous posts that there is only a weak correlation between the amount of money a Virginia school district spends per pupil and educational achievement as measured by the pass rates on Standards of Learning tests. But there are other ways to make the same point.

Our Lynchburg correspondent Jim Weigand has brought to my attention the publication of a report  based on 2015 data comparing the Required Local Effort (RLE) for Virginia’s localities and what the school districts actually spend on K-12 education.

For those of you not conversant with educratese, RLE represents the minimum expenditure required to meet the state’s “Standards of Quality,” an assessment of inputs such as the ratio of teachers and staff to the number of students. By comparing RLE to actual local expenditures, the VDOE report calculates the amount of a jurisdiction’s own tax dollars (as opposed to state and federal dollars) that it spends above and beyond the minimum requirement.

As the chart above shows, school districts vary widely in how much fiscal effort they devote to funding their schools — from 7% above the rock-bottom minimum in Patrick County to 284% of the minimum for the town of West Point. And what is the payoff for that extra spending? We compared the RLE percentage with average SOL scores for the 2014-2015 school year for each school district.

SOL_pass_rate

As can be seen in the scatter graph above, there is almost no correlation at all. The R² measure of correlation is less than 5%.

Does that mean the extra money is wasted? Not necessarily. Some school districts may be spending more money because they have a higher percentage of students who are handicapped, economically disadvantaged or speak English as a second language. But the graph is a pretty good sign that dumping more money into schools to meet or exceed the Standards of Quality is not an effective strategy.

Instead of blindly plowing more money into Standards of Quality, perhaps money should be steered to schools with more at-risk students. Or perhaps we could study what the successful schools are doing differently from the less successful schools and try to replicate the secret sauce. Or perhaps we could do almost anything but what we’re doing now.

To see the numbers for all localities in Virginia, click here.

Update: Over at Cranky’s Blog, John Butcher makes some adjustments to the data that I should have made, breaks down performance by SOL subject matter, and offers his own unsurpassed commentary.

Share this article

ADVERTISEMENT

(comments below)

ADVERTISEMENT

(comments below)

Comments

16 responses to “Another Useless Educational Metric”

  1. LarrytheG Avatar
    LarrytheG

    you must be having problems here – the prior blog post seems to be having trouble appearing and retaining comments made.

    but I’ll make the point again- this is not about the SOQ and match.

    you’re equating money above the SOQ as being spend for some constant value of education and that’s simply not the case because schools spend the discretionary money above required effort – on different things – of which almost none of them actually document specifically.

    so – one school could spend heavily on at-risk or other high value academics and another could be spending it on perks and goodies for the college bound – at the expense of K-3 programs to deal with at risk kids.

    you simply do not know – so you cannot compare and what you are seeing is not going to have any relevant correlation between money and overall school academics.

    but I LIKE the direction where you are headed – but we’re never going to get there if you can’t see what exactly the local effort over and above the required match is actually spent on – specifically.

  2. Cville Resident Avatar
    Cville Resident

    The ridiculous metric is to use SOL pass rates. They measure nothing at this point. A more interesting metric would be a district’s students graduation rate from 4 year colleges or trade programs. These stats don’t really address the efficacy of spending on education. They do highlight the uselessness of SOLs.

  3. Spending More But Not Teaching More – CrankysBlog

    […] now has published the 2015 RLE data.  The estimable Jim Bacon yesterday posted an initial look at those data, which prompted me to take a more detailed […]

  4. Les Schreiber Avatar
    Les Schreiber

    Having been a school teacher for “a few years”,I believe the the early background that the child grows up in is of critical importance. Do the parents read to the child and stress at home and the importance of academic achievement. This is becoming even more important as the importance in the job market for tech skills grow. The earlier poor kids get this type of exposure the better.
    Having kids start earlier I believe to be as part of the answer.
    As to spending,over the past six years,teachers have experienced a decline in real purchasing power as wages have virtually stagnated.How do you attract tech knowledge professional to a “profession”that offers a declining standard of living.

    1. Cville Resident Avatar
      Cville Resident

      Les,

      Very true about the parents. In this day and age, upper middle class parents’ kids are dominating. Those parents inculcate learning in their kids from a very early age. If we were to look at the life outcomes of the class of 2000 from SWVA and compare it to suburban Richmond, I guarantee the results would be laughable when you examine the separation. In the hiring I’ve done, I see plenty of kids out of U.Va., William and Mary, JMU, and Virginia Tech. I see a heck of a lot of kids who grew up in the Urban Crescent and the Charlottesville area that graduated from those schools. I almost never see a kid from the rest of the state who went to those schools. This is real life, not some ridiculous SOL test. You want a kid to succeed in this day and age? Move to an affluent suburban county, stress academics and reading from an early age, live around other parents with the same values….the schools that they attend will reflect those values. And guess what? Those schools’ parents pour the resources/taxes into them. I can name you the top 20 high schools in this state, and I’m pretty sure most people on this blog would name the same 20. This isn’t rocket science. These ridiculous graphs prove absolutely nothing. Look at life outcomes and then get back to me.

      1. James A. Bacon Avatar
        James A. Bacon

        Cville, What this “ridiculous graph” proves is that fiscal effort (a proxy for money) has minimal correlation with SOL outcomes. What other “ridiculous” graphs published on this blog have shown is that roughly 50% of the variation in SOL pass rates can be attributed to the socio-economic status of the students.

        If one were to follow your logic, very little can be done by the schools to improve the outcomes of less affluent students — the outcome is pre-determined by their affluent environment!. But the fact is, schools and teachers do have some influence. They can’t perform miracles, but some schools do a better job than others, even when adjusted for socio-economic factors. What are those things? We need to find out more. But throwing money at the problem isn’t one of them.

        1. Cville Resident Avatar
          Cville Resident

          I don’t disagree with what the graph shows. But SOL scores are laughable as a point of “proof”. I’d care a lot more about the life outcomes of my kids than a test (SOLs) that affluent school districts regard as a joke.

          1. James A. Bacon Avatar
            James A. Bacon

            Cville, I agree that SOLs are a flawed tool, especially for measuring the educational achievement of the better students. But I’m not sure what you’re saying about the connection between SOLs and outcomes. Are you suggesting that some students who fail SOLs might turn out to be educational high achievers, or otherwise successful in life?

    2. TooManyTaxes Avatar
      TooManyTaxes

      Les, I agree about the need to pay experienced and good teachers more. We have a problem keeping experienced teachers in Fairfax County. The Superintendent has proposed larger raises for those who have taught for 5-20 years. But she isn’t serious since she’s also proposed to increase pay almost comparably for staff. How many staff are leaving? Hardly any according to some teachers I know. I’d hold pay of anyone not in the classroom to 1% and move the money to teachers. But it will never happen.

  5. Cville Resident Avatar
    Cville Resident

    No, but I am suggesting that “passing” the SOLs is not predictive of success in life. I would bet a thousand bucks that the same kid placed in western Henrico v. Galax is almost certain to have a much different life outcome regardless of SOL passage. The entire culture in western Henrico (including the public schools) is driven by achievement. SOLs are laughed at by a lot of people (parents, teachers, and admins) in that area. You’d be surprised at the curriculum at certain schools (actually quite good).

    1. James A. Bacon Avatar
      James A. Bacon

      Are SOLs used as predictors of success in life? I thought they were used as measures of academic performance — predictors of a student’s ability to master basic material (not advanced material), progress academically and graduate from high school.

      1. Cville Resident Avatar
        Cville Resident

        I don’t think the SOLs are measures of academic performance. They’re measures dictated by politicians.

        I’m more interested in project-based assessments or assessments of critical thinking. Those assessments are better measures of “academic performance.” If you were to look at schools in the Urban Crescent or Charlottesville/Albemarle, you’d find some interesting curriculums that are applicable to the 21st century. A lot of other school districts? You’d find SOL heavy curriculum.

        Let’s be honest about what the SOLs were, are, and will always be….in the 90s, a group of politicians thought that they could hold “poor performing” school districts “accountable” through test scores. The SOLs measure retention of content (memorization of facts and concepts). They do not measure cognitive abilities, metacognition, or critical thinking (what one would hope that a child might learn for an age of constant change). What SOLs really measure is how much time and emphasis a particular school district places on memorization of facts. Whether we’re talking Plato or Steve Jobs, no one believes someone is “competent” or “academically on track” just because they can memorize something that is drilled into their heads over and over again.

        We live in an age where we want people to be adaptable, mentally agile, and critical thinkers. Using the SOLs as a metric of academic performance is about as useful as relying on phrenology!

        1. LarrytheG Avatar
          LarrytheG

          it’s not just about those who go to 4 year college. The basis of public education is an employable workforce and that metric is a basic 21st century education that allows one to do 21st century work including work that does not required a 4yr degree.

          education is about NOT paying for entitlements as much as it is about the personal edification of our own offspring…

          1. Cville Resident Avatar
            Cville Resident

            I completely agree. That’s why I am saddened when I see people take the SOLs seriously. Whether it’s a 4 year degree or a trade certification, part of education should be focused on skills, whether manual or mental, rather than content regurgitation (SOLs).

  6. LarrytheG Avatar
    LarrytheG

    this is foolish. Why does anyone presume that extra, discretionary money, over and above the required SOQs goes for something you think it goes for?

    how do you KNOW what that money -over and above what is required actually goes for SOLs OR early childhood or WHAT?

    why would any presume what it goes for – or not if you do not know?

  7. LarrytheG Avatar
    LarrytheG

    re: ” That’s why I am saddened when I see people take the SOLs seriously. Whether it’s a 4 year degree or a trade certification, part of education should be focused on skills, whether manual or mental, rather than content regurgitation (SOLs).”

    then how or what do we measure to assess competence ?

    I’m also opposed to high stakes testing and a proponent of more informal assessments but I also believe you must measure performance and competence if for no other reason to know what the kid has NOT YET mastered and needs more help on.

    if we don’t measure and provide the help needed – how will people graduate and have the basic education to even get an occupational certificate?

    and keep in mind – whether you go into the armed forces, or get a Doctors or Engineers license or even an X-ray tech or law enforcement certificate it DOES REQUIRE a “high stakes” test and truth be known most of us don’t want to be treated by a Doc or Engineer that failed their license and board knowledge requirements.

Leave a Reply

Read More