Monday, January 23, 2012

Questions from a data flub

All of us who work with numbers know how easy it is to make a mistake.

Charlotte-Mecklenburg Schools made one last week when it posted inaccurate (and still unexplained) results for a new on-time graduation-track calculation,  then was slow to recognize the error.  Late Friday,  interim Superintendent Hugh Hattabaugh pulled the school progress reports offline, telling the school board in a memo that "several issues of data accuracy have come to light."  The reports will be reposted Feb. 3 "after the data has been fully audited,"  he said.

The incident poses some serious questions for a district that prides itself on being data-driven on everything from education strategies to accounting for public money.

Some numbers lend themselves to a common-sense reality check.  It's like stepping on a scale: If it's five pounds off, you might believe it.  If it's 50 pounds off you know the scale is broken.  For anyone familiar with high schools,  numbers showing fewer than 2 percent of all students have ever flunked a grade are a clear signal that the scale is busted.

Hattabaugh and his officials are facing questions about why they didn't catch the problem.  But I'm wondering about principals who got their school reports almost two weeks before the error went public.  At many of those schools,  the bogus numbers were wildly out of sync with reality.  Did anyone say  "Hey, this can't be right"?  If not,  are principals so overwhelmed by central-office data that they've stopped caring whether numbers are accurate and meaningful?  If so,  were they unable or unwilling to let their superiors know there was a mistake?

I asked Hattabaugh about that during the weekend school board retreat.  "They were probably just thrilled that it was a good number,"  he said, smiling.

I don't know if he's right.  But if his school leaders are happy with glowing but false data,  the problem goes deeper than a central-office flub.

No comments:

Post a Comment