by James C. Sherlock

Many dramatic changes in school policy are being made and contemplated by the Virginia Department of Education. The references justifying the changes are inevitably academic “studies.”  

I’m sorry, but academics can and do design studies to provide whatever results they are looking for. That is why so many are not replicable – much like push polling. Tailor the inquiry with only the carefully worded questions that will yield favorable results. Delete any that may not support the objectives.

I often wonder if VDOE conducts any original research using Virginia data to support the changes they implement. If they do, I have never seen it.

VDOE has massive decision support databases. I use them regularly. One of the newer data troves is the 2020-2021 Virginia Educator Ethnicity and Race data.

I decided to run an experiment to prove the case for such research.

An experiment. I used that database and others published by VDOE to test the theory that matching teacher demographics with student demographics yields better minority student performances.  In my very small and statistically insignificant sample, it did not. 

But this drill itself shows what can and should be done by VDOE at scale.

The Educator Ethnicity and Race data show that 80% of Virginia public school teachers are white. They also show that eight districts have fewer than 50% white teachers. They are:

• Norfolk – just less than 50%
• Surry County – 43%
• Richmond City – 42%
• Franklin City – 40%
• Portsmouth City – 35%
• Brunswick City – 25%
• Sussex County – 23%
• Petersburg City – 18%

To give a mental weighting to those percentages, there were 3,672 other than white teachers and 2616 white teachers total in those eight school divisions, or 42% white. In total number of black teachers compared to white teachers, the totals were 3,374 black and 2,616 white teachers across the eight districts.

As a reference for the poverty of the students, I added the percentage of free lunch students from yet another VDOE data base.

• Norfolk – 59%
• Surry County – 50%
• Richmond City – 68%%
• Franklin City – 88%
• Portsmouth City – 62%
• Brunswick City – 64%
• Sussex County – 61%
• Petersburg City – 75%

Then I added the 2018-2019 math and reading SOL pass rates for black students.

• Norfolk –                   math 63%; reading 57%
• Surry County –         math 76%; reading 74%
• Richmond City –      math 50%; reading 50%
• Franklin City –         math 66%; reading 65%
• Portsmouth City –   math 71%; reading 66%
• Brunswick City –     math 64%; reading 57%
• Sussex County –      math 80%; reading 76%
• Petersburg City –    math 52%; reading 51%

For this experiment, I sorted and re-sorted the spreadsheet with these data to see if I could find correlations among the percentages of non-white teachers, the poverty of the students and the performance of black students on SOLs.

Findings. The math SOL pass rates among black students in these districts varied from a high of 80% in Sussex County to a low of 50% in Richmond. Reading pass rates varied from 76% in Sussex to 50% in Richmond.

Two correlations were noted off the bat:  

• Pass rates for math and reading correlated directly and uniformly from top to bottom. meaning that the black students in the school district who had the third highest math pass rates also had the third highest reading pass rates; etc.
• Pass rates for black students in math matched or exceeded their pass rates for reading in every district.

Student poverty was generally a predictor of black student SOL performance among these eight school districts. The higher the poverty rate the lower the scores.  

But there was an outlier. Franklin City students had a poverty rate of 88%. Yet Franklin black students outperformed black students in Norfolk. The overall student poverty rate in Norfolk was 59%.

The percentage of white or black teachers in these eight schools did not correlate to student SOL performance.

The three largest of those eight districts by far were Norfolk, Richmond and Portsmouth in that order. Looking at those three in isolation, the best black student SOL performances by a wide margin were in Portsmouth, the smallest of the three school districts and also the one with the smallest percentage of white teachers (35%).  

However, this was not a trend even among the three districts because of the terrible performance of black students in Richmond.

I went through this drill not to develop a statistically significant study, and this is not one, but to demonstrate the kind of analysis that is possible with the resources available to the Virginia Department of Education.  

After the statistical analysis, VDOE should examine the outliers to determine what lessons might be learned there as well.

Recommendation. Such analyses completed by VDOE provided to the public in support of policy changes would greatly increase confidence in their policy prescriptions.

I recommend they conduct such assessments to test their policy assumptions and then run field trials to prove those prescriptions actually work before levying the huge and unknown costs of major changes on the entire Virginia public school system.