Data on death and the death of data

On PM tonight, the Scottish Information Commissioner was interviewed about the publication of survival rates for patients who are operated on by surgeons across Scotland. (It's not on the website yet but Eddie Mair seemed to know all about it.) Apparently some journalists asked for it, and he saw no reason why they should not be given it. So now it's out there, in the public domain, and although no 'league tables' will be produced, it only takes MS Excel to create one. So how long before surgeons, like teachers, start to see the people who pass beneath their scalpel as part of a statistical game? I have often found myself looking at my A-level groups and calculating how many can leave college before my percentage retention rate dips below the national benchmark. How many of the C/D borderline GCSE pupils have to get a C to nudge the school passrate ahead of the other big schools in the area. Do you really want your surgeon's advice to you to be tainted by her having one eye on her running average for the year? Because the media and the watchdogs and the government are not interested in the subtle reasons that students leave college or kids fail GCSEs or patients die on the operating table, they are interested in the headline figure.

Data is a lot like humans: It is born. Matures. Gets married to other data, divorced. Gets old. One thing that it doesn't do is die. It has to be killed.
Arthur Miller
(from the wonderful page of quotes at sysprog)

Now those Scottish death rates are out in the world of data, they will never die. They will remain as a lazy person's way of evaluating the health service, much as school league tables are a lazy person's way of evaluating schools. I remember once when the head of History at a school where I worked made a mistake in calculating his pass rates for his yearly exam meeting with the Head. The Head was horrified and sprung into action, labeling the department as a cause for concern and talking at length about it at the next management team meeting. When the error was discovered it took several more meetings and a formal 'setting things straight' from the Head before the idea that History was a department in dire straits was replaced with a truer picture.

And now I come to write this, I realise the same is true of the labels we apply to kids.