Reform the SOLs, Don’t Kill Them

Cranky begs to differ with the Big Bacon.

Honorable Sir!

Your Jeremiad on the Standards of Learning testing makes five basic points and concludes that it’s time to “scrap the SOLs and move on.” I’d like to mostly agree with your five points and suggest some different conclusions.

SOL Results Have Guided Home Purchase Decisions, With the Effect of Harming Low-Performing Schools

That is almost certainly true.

There is a plethora of Web sites dedicated to reporting score results – even one from the Department of Education. And surely one reason there are few school-age children in my Richmond neighborhood and many in your Henrico enclave is the quality of the schools.

Would you then have parents rely upon word-of-mouth rather than actual data in selecting a place to raise the family?  Would you conceal the performance of this appalling middle school from the parents who are paying the taxes to support its gross failure to educate their children?

It’s not the job of parents to harm their kids by sending them to an awful school in order to improve the school. It’s the job of the school board and the Board of Education to fix the awful schools. And it is far past time to hold the school, the school board, and the education board accountable if they don’t deal with problems such as the example in the link above.

To do that we must have a quality measurement. As one of your commenters pointed out, the education establishment loves to measure inputs: money, facilities, credentials. But those things don’t measure productivity. For sure the SOL is imperfect but it’s the only productivity measure we have at hand. We should be thinking of ways to deal with the imperfections, not relapsing to a system where the only quality measures do not in fact measure quality.

The SOL Penalizes Poverty

It’s clear that children from economically disadvantaged households underperform their more affluent peers on the SOL tests. It’s also clear that some school systems with large ED populations perform better (or worse, e.g., Richmond) than others.

Thus, the bare SOL pass rate is not a fair measure of school quality.

Should we then abandon measurement of educational quality or should we look to improve the measure?

If fact, we have a better measure that is essentially unaffected by economic disadvantage, the Student Growth Percentile. The State tells us:

A student growth percentile expresses how much progress a student has made relative to the progress of students whose achievement was similar on previous assessments.

A student growth percentile complements a student’s SOL scaled score and gives his or her teacher, parents and principal a more complete picture of achievement and progress. A high growth percentile is an indicator of effective instruction, regardless of a student’s scaled score.

And a low SGP tells us that the kid didn’t learn much in comparison to the other students who started in the same place. See this for a discussion of the way the SGP illuminated the awful productivity of some of Richmond’s teachers.

Yet our Board of “Education” has abandoned the SGP.  See this (scroll down to Part F) for a discussion of their bogus reasons.

Bottom line: We know the SOL is unfair to less affluent kids. We know how to correct for that effect but our Board of “Education” doesn’t want to use the tool that does that.  Is that a problem with the SOL or a problem with the Board?

Cheating

There is a long and ugly history of Virginia schools cheating to improve SOL scores. See this for a particularly cynical example. Go here and search for “cheat” to be inundated with data on the subject.

But is this a problem with the SOL testing or with the schools and the Board of Education?

If we take it that it’s crucial to have a measure of educational output, then we’ll have to be prepared to deal with attempts to skew the data. The (unacceptable) alternative is to forget about measuring teaching quality and to let far too many children be harmed by inadequate teaching.

Teaching to the Test

You, and others, condemn the SOL because it encourages teaching to the test. In fact that is not a problem; it is part of the reason for having the SOL.

Consider the alternative: If every school and, in some measure, every teacher gets to decide what shall be taught and what shall be on the exam, there can be no uniform measure of educational quality and no accountability (there’s that word again!).

Indeed, the Department of Education literature is ripe with discussion of “aligning the curriculum” to the Standards of Learning.  Thus, if we have appropriate standards, we get schools that teach the appropriate material.

In terms of your example: If we think irregular polygons are important, we can put them in the standards and in the test; that will assure that the schools will try to teach irregular polygons.

In short, teaching to the test is not a bug; it is a feature.

Accountability remains elusive.

I like to talk about the argyle sock effect: If you go into an organization and find the employees wearing argyle socks, you know that the boss wears argyle socks.  First corollary: You know it’s a good grocery store if you see the manager handling checkout or corralling carts.

The first step for improving education is evaluating how well the teachers teach.  Yet the current system does not work and the Board of “Education” has abandoned its tool, the SGP, that can work.  Indeed, their standards for evaluation are a crock.

The Board of Education is wearing white socks with its dress shoes.

Accountability starts at the top: If the Governor were serious about improving the schools, he would fire the current members of the Board of Education and replace them with people who understand accountability.

The Governor has not done so.  The Governor is wearing white socks with his dress shoes.

Conclusion

As I said at the start, you have correctly identified major flaws in the current system for measuring student achievement. But I think your solution would take us back to the Bad Old Days when there was no objective measure that could allow us to hold public school teachers accountable for the job they do with our children.

That said, thanks for shining some Bacon’s Rebellion light on this important subject.

John Butcher publishes Cranky’s Blog and is a frequent contributor to Bacon’s Rebellion.

There are currently no comments highlighted.

6 responses to “Reform the SOLs, Don’t Kill Them

  1. Well.. it appears that Cranky has some ideas beyond pure criticism …..and is “schooling” Bacon!

    re: ” SOL Results Have Guided Home Purchase Decisions, With the Effect of Harming Low-Performing Schools”

    As pointed out.. all 50 states now provide school academic performance data (per No Child left behind) – and boundaries are obtainable also.. so 3rd party folks go get that data from various places and put it together in a database/website.

    the interesting thing to me is that some of the data reported to the Feds and grabbed by the 3rd parties to aggregate is not easily located at VDOE, or the School District or individual school:

    Take a look at the Niche list of Elementary Schools in Henrico that show the SOL performance, student-teacher ratio, percent of free/reduced, and percent of teachers in their first and second year…

    https://www.niche.com/k12/search/best-public-elementary-schools/d/henrico-county-public-schools-va/

    this is all data that the schools have to report.. but try finding it on the Henrico County school district and individual schools websites.

    re: Cheating and Teaching to the test…

    VDOE – to it’s credit is changing the way it tests in the critical early grade SOLs.. it’s a much better way to test.. it calibrates where the kid is much more precisely – and it tests more frequently.. more casually.. The data collected can show a timeline for the kid for how much they are learning during the school year.. where they need help… and that data constitutes the growth of each kid – and the classroom as a whole.

    The focus is on improving the performance of the kids – and the teacher in doing their job.. rather than a process for tarring “bad teachers”.

    It’s not only a more fair system – it recognizes that there is no such think as a “good” or “bad” teacher in a binary way.. that .. all teachers – are in a continuum… as they get better at what they do.. by learning what works with what kids and what does not.

    The whole idea that some grad out of College comes fully-equipped as a top-notch teacher is simplistic and really – stupid. More than many other jobs.. teaching is learned “on the job” and very few come out of college to perform at high levels.. and a system that is focused on ferreting out “the bad ones” ..is an ignorant system that harms both teachers and kids..

    the goal is for the teacher themselves to learn and get better – to improve their own competencies..

    yes… at some point -you have some that just are not cut out to be teachers and should seek other work.. but a system that is set up to foster failure as some kind of “accountability” just shows us how misguided people outside the profession are.. and how harmful simple-minded perspectives are…

    Taking brand new teachers out of College and sending them to the toughest schools to teach is not smart.. but it’s what many school districts (even good ones) do because veteran teachers typically can pick where they want to teach and they won’t pick the tougher schools where the current “accountability” process can effectively damage or end the careers of teachers who would be highly rated at schools where kids are easier to teach.

    • I agree testing should occur more than once per year per subject, most especially in the earlier grades. But as students get older, they need to understand life is full of one-time, all-or-nothing tests. I had a professor in law school who had one test for a nine-month course. Every other section had several tests. Preparing for it was hell. But that happens in life. I see no reason why juniors and seniors in high school shouldn’t face a single SOL.

      And some subjects are more fact-driven than others. For those courses there are right and wrong answers. For others, maybe a range of answers or a best answer justified by the student’s logic and mastery of material facts.

  2. I am late to the discussion, but I would like to offer a suggestion that is different than making adjustments at the margin of a system that is not preparing our young people for the future.

    We are discussing Standards of Learning but as with many topics today the name is misleading. We are assessing standards of teaching, or more accurately Standards of Regurgitating. We are measuring students’ ability to remember and regurgitate facts.

    The Chinese educational system, at its highest levels, produces students that are far better at doing this than American students, but are also markedly lacking in critical thinking skills and their ability to create new ways of doing things. They are so indoctrinated with the idea of “one right answer” that they are unable to put together what they know in new ways or to create new insights. This is limiting their progress as a society, just as we are here.

    Probably the students who are the best at regurgitating facts are those who go to medical school. In Med school, they are taught many more “facts”. After residency, they are fully skilled at managing the symptoms of chronic disease while their patients get progressively less healthy. They are taught little to nothing about how to help their patients live full, healthy lives.

    My point is, we can design educational systems that are more efficient at climbing the ladder. But it is very likely that the ladder is leaning against the wrong wall.

    Why are kids so inquisitive in their early years, but either complete dullards or skilled automatons in their later years?

    Robert Kiosaki once commented on those that were smart enough to game the system and pass, and also smart enough not to succumb to it. He said, “As entrepreneurs, the “C” students hire the “A” students, and the “B” students work for the government.

    Education, to me, is a “push” activity. We are trying to push standard methods of behavior and believing into the mass of our population so they can fit a mold and be successful cogs in the machine of our current culture.

    Learning is a “pull” activity that starts with the innate curiosity that is within us all, to a greater or lesser degree, until we stamp it out of most students. Everyone else is just trying to get a job.

    Methods of assessment are useful. But we tend to get what we measure. If we measure conformity, we get conformity, even with under-performers. We need to nurture innovative thinking, confident trial and failure, collaboration, multiple answers, better questions.

    We would be better served to reevaluate what we are trying to achieve and develop the appropriate means of assessing how well we are achieving those ends.

  3. Among the many comments this excellent discussion could provoke: you say, “teaching to the test is not a bug; it is a feature.” Yes; but then, it is apparent that SOL testing must cover ALL the important elements of secondary education in order to have the desired effect. To that end, Virginia’s recent demotion of “history” to non-tested status (e.g., http://wapo.st/2xHWZNM) only contributes to the widespread popular ignorance of, and failure to participate in, what we used to call local “civic” activities and institutions. Not to mention, our vulnerability to politicians who shamelessly promise what common sense abhors and our Constitution forbids. Bring back history testing.

  4. I’m not a proponent of “regurgitation” but in the lower elementary grades… core proficiencies are critical to being able to “learn” later.

    Critical thinking, problem solving, being able to article your own ideas and collaborate with others – is not going to happen if you are not competent at reading, writing and mathematical literacy.

    The problem with the idea of testing more than reading, writing and math is that it takes away from those core competencies.. for some kids .. that must learn the basics is that history is pretty much worthless if you are a poor reader… most all other subjects are similarly not useful if basic competencies are not attained. It’s a building block process.. and you have to get the basics before you can do the rest.. and if you are having trouble reading.. you need that extra time for more reading -not history.

    For every kid that is on free/reduced lunch, lives in poverty circumstances, is “at risk”, .. it’s the basics that are the hardest and the most important and that’s why you see Reading and Math highlighted..

    Take a look at Shady Grove Elementary School in Henrico:

    Academics
    Percent Proficient – Reading 97%
    Percent Proficient – Math 97%

    now take a look at Montrose in Henrico:

    Academics
    Percent Proficient – Reading 52%
    Percent Proficient – Math 62%

    https://www.niche.com/k12/montrose-elementary-school-richmond-va/

    history is not going to help those kids at Montrose…

  5. Cranky –

    Thanks for eating Big Secondary ED alive, its masters and knaves, and for taking big hunks out outta cabals of local dead-weight bureaucrats, their dissembling and knavery that often hide the truth of what’s happening to kids supposedly under their care, while disguising their system’s failures as good public policy that advances kids’ education, instead of the plain truth that the system far too often thwarts the education and future of generations of kids, a fraud and tragedy imposed on innocents, their parents, us taxpayers, our society, and our neighborhoods.

    And for jerking Big Bacon outta his ditch, back onto the straight and narrow.

    From now on and forever after too, I am going to keep a sharp and wary eye out for socks and shoes – socks and shoes of all sorts and colors, how they mix and match, making fools of one another, and their showy masters too.

    And thanks for doing it all for the kids now and forever. You’ve been added to my official life time list of heroes.

Leave a Reply