Bad Statistcs in the Irish Times

Sean Flynn has a front-page piece in the Irish Times entitled Number of firsts at DCU doubles in less than decade which has a very interesting point about grade inflation and then proceeds to make some very silly conclusions from the statistics on offer.

The accompanying table shows universities have a higher proportion of students getting honours degrees and concludes that "When figures for both firsts and 2:1 honours degrees are combined, UCC and Trinity emerge as the universities where students appear to have the best prospects". This is plain wrong. Yes, these two universities have the highest proportion of students getting good degrees. Yet these statistics say nothing about the ability of candidates on entry or the distribution of grades in the courses the universities have to offer. Arts and social science subjects are generally harder to fail yet harder to excel in while the distribution for science and engineering subjects is flatter with both more firsts and more thirds. Hinting to potential entrants that they'll do better at one institution over another (as the piece insinuates) is simply misleading.

Also Flynn notes that UCD fails to return a full set of results to the HEA yet concludes that: "13 per cent gained a first and 26 per cent a upper second-class honours (2:1) degree." even though the percentages in the accompanying table fail to sum to 100%. Help anyone?

In reality the quality of degrees is unlikely to vary very much from university to university in Ireland. Why? It's a small country so there's only one place to study some subjects. How can you judge UCD's veterinary graduates and Trinity's pharmacy graduates, even against each other? You can't as they're the only places in the Republic to study the subjects. Identical salary scales, equal capitation and little university autonomy ensures that the 'product' on offer in Irish universities is going to vary very little. Policy has always been focussed on ensuring breadth of subjects and wide access and sometimes quality at undergraduate level has suffered as a result.

I'd like to see a serious study (a bit like this one) of university gradings controlling for ability of entrants, drop-out rates and the type of course mix on offer. Until that's done there's very few conclusions (and certainly no front-page ones) to be drawn.