The Problem with “Student Engagement” Statistics

This table shows the number of schools falling into each accreditation category. Data source: Virginia Department of Education. Note: Dropout rate and Graduation & Completion rate apply to high schools only.

Under the new public school accreditation standards, the Virginia Department of Education (VDOE) has begun tracking chronic absenteeism, the dropout rate, and the graduation & completion rate. The merits of these seemingly common-sense metrics is that (1) they are easy to collect, and (2) they measure very important things. In an ideal world, we’d like to see everyone graduate from high school, so we’d like to see the dropout rate go down and the graduation & completion rate go up.

The table above, taken from a VDOE press release issued today, shows the number of schools meeting accreditation standards for the three metrics (Level One), those that are near meeting the standards (Level Two), and those that fall short (Level Three).

While seemingly objective, these three metrics are very different from academic-performance metrics based on Standards of Learning (SOL) tests, which have gone through a rigorous design and approval process, and the testing of which is essentially audited and policed by VDOE to prevent cheating. While far from perfect — the criticisms of SOLs are many — the SOL numbers are reasonably trustworthy as indicators of students’ basic proficiency in reading, math, and science.

The student-engagement metrics are far easier for school administrators to manipulate. As I have noted previously, the pressure is intense for school districts, and individual schools within those districts, to show improving numbers. But administrators are dealing with the intractable reality of social breakdown, the dysfunctional culture of poverty, and blowback from schools’ own social promotion policies. The fact is, large numbers of high school students are unwilling to attend school or take classroom instruction seriously.

In the brave new world of Virginia education policy, school districts devote resources to rounding up truants and bringing them to school — all with the goal of reducing chronic absenteeism numbers. But dragging a kid to school doesn’t mean he (or she) will attend, much less participate in, every class — especially if they are so far behind that they cannot follow the discussions or hope to complete the work. If their classmates are lucky, the problem kids will just sit in the back of the class, fiddle with their smart phones or otherwise tune out. If the classmates are unlucky, the problem kids will act out and disrupt the class. Under the court-enforced disciplinary regimes in many school districts, teachers can no longer eject problem students without first engaging with them, reasoning with them, and trying to coax them into behaving — all of which takes time away from classroom instruction.

The new accreditation push is for schools to demonstrate that students are showing “progress” even if they fail to meet SOL proficiency standards. In grades 3 through 8, in which students take SOL tests every year, it is possible to show that a student, though still falling short of state standards, falls short by a smaller margin than the year before — thus showing “progress.” By contrast, I don’t know how it’s possible to objectively demonstrate progress in high school. If high schools rely upon teacher evaluations, then the process is easy to corrupt by leaning on teachers to fudge their judgments.

Given the contradictions and perverse incentives within the system, this is what I predict will happen: At many schools, we will see improving statistics for “student engagement” even as educational quality declines. More students will complete high school but the perceived value of a high school diploma (or certification of completion) will decline in the marketplace as employers confront the reality that increasing numbers of students are graduating high school without achieving mastery of basic skills. Those who are, in effect, socially promoted out of high school will be deceived to think they have skills that they do not. And those who worked hard to acquire those skills will see their diplomas devalued.

I’m not sure how we measure the declining value of high school diplomas, so it may be a long time before perception catches up with reality. But I fear a lot of damage will be done in the interim.


Share this article



ADVERTISEMENT

(comments below)



ADVERTISEMENT

(comments below)


Comments

7 responses to “The Problem with “Student Engagement” Statistics”

  1. Reed Fawell 3rd Avatar
    Reed Fawell 3rd

    What does the chart at the top of this post tell us?

    1. The table shows the number of schools falling into each accreditation category. I have updated the cut-line to make that clear.

      1. Reed Fawell 3rd Avatar
        Reed Fawell 3rd

        Actually, a powerful case can often be made that a school with higher dropout rates and absenteeism is a better school for the great majority of kids in that school, than many schools with a higher dropout and absenteeism rates.

        In fact, quite often, these sort of statistical studies are done in order to hide real and festering problems in education in schools, and to instead use those studies to disguise those hidden problems as signs of progress.

        And far too often otherwise responsible leaders go along with this sham so as not to rock the boat. Meanwhile kids who want to learn or otherwise would learn suffer the consequences.

        Jim, you suggest the possibility of this in your commentary. I want to highlight it. I suggest studying the wrong statistics delays the solving of real problems instead of false ones.

        1. Reed Fawell 3rd Avatar
          Reed Fawell 3rd

          Correction to 1st paragraph above:

          Actually, a powerful case can often be made that a school with higher dropout rates and absenteeism is a better school for the great majority of kids in that school, than many schools with LOWER dropout and absenteeism rates.

  2. LarrytheG Avatar

    re: “The fact is, large numbers of high school students unwilling to attend school or take classroom instruction seriously.”

    Are we sure of this? It’s an easy claim but what does it really mean in real numbers and even then – …..

    I’m not sure what the point is here to be honest as it seems to present a binary choice between allowing disruptive students to disrupt classrooms and to harm others education OR to be tossed out altogether since they will “never” lean anyhow…

    There are in most school districts alternative education for kids who do not fit or belong in a normal classroom.

    Here’s an example: https://www.spotsylvania.k12.va.us/domain/1583

    I think this is appropriate – and it does, in reality, provide targetted resources for the ones that can be helped .

    I think this is a better way that the binary choices and anyone who knows young people – knows that it can take some time for some kids to get on track and it was not that long ago that some kids needed to go into the military to get what they really needed to get themselves straight.

    The point here is that we do not give up on kids. even disruptive ones. More than a few with the right kind of environment will get back on track and go on to become productive adults and the ideas that some have to just abandon them – well those folks certainly do not belong in education.

    1. What binary choices are you talking about? Who is saying we should “give up” on disruptive kids? I’m talking about acknowledging the reality of what’s going on in schools today — not “giving up” on disruptive kids. Looks like another one of your straw man arguments.

      1. LarrytheG Avatar

        no strawman… exactly what are you really suggesting here? :

        ” devote resources to rounding up truants and bringing them to school — all with the goal of reducing chronic absenteeism numbers. But dragging a kid to school doesn’t mean he (or she) will attend, much less participate in, every class — ……… If the classmates are unlucky, the problem kids will act out and disrupt the class. Under the court-enforced disciplinary regimes in many school districts, teachers can no longer eject problem students without first engaging with them, reasoning with them, and trying to coax them into behaving — all of which takes time away from classroom instruction.”

        what’s the point of this dialogue? It sounds like you don’t think it’s the right answer to try to keep these kids in school.. that it won’t help them and it will hurt others they get imposed on.

        correct?

        it’s actually NOT a binary choice as presented. There are alternative education programs where disruptive kids are sent – at some schools.

        Have you acknowledged this as a legitimate 3rd option between the binary knife-edge approach?

        I posted a link to an alternative school – I’m quite sure it is not perfect and even then there are the tough nut kids that don’t get a diploma but it’s simply not true that the other kids in the schools get a degraded diploma because they did not separate the disruptive kids.

        The premise presented – that trying to keep disruptive kids in school – is a choice between imposing them on other kids and harming or expelling the disruptive kids is a binary choice that is not the reality. There is data to back up that premise, in part because some schools have the 3rd option to separate but beyond that, some of these schools have large numbers of harder-to-teach low-income students have trouble even when not disrupted.

        But what is the point of the blog post to begin with when it concludes like this:

        ” More students will complete high school but the perceived value of a high school diploma (or certification of completion) will decline in the marketplace as employers confront the reality that increasing numbers of students are graduating high school without achieving mastery of basic skills. Those who are, in effect, socially promoted out of high school will be deceived to think they have skills that they do not. And those who worked hard to acquire those skills will see their diplomas devalued.

        I’m not sure how we measure the declining value of high school diplomas, so it may be a long time before perception catches up with reality. But I fear a lot of damage will be done in the interim.”

        where do you get this from guy?

        this is a broad brush statement that seems to imply that because we don’t expel disruptive kids that we are destroying the value of diplomas…

        geeze… baby – bathwater… we’re all gonna die because of these disruptive kids… let me guess.. you got a sign on your front lawn that says ” Get off my lawn”… right? 😉

Leave a Reply