This piece by Philip Eden from the Daily Telegraph (7 Feb 2009) was brought to my attention by Rev Philip Foster of Cambridgeshire:
"How a few statistics snowballed into a lie"
By Philip Eden
"THE heaviest snow in Britain for 18 years? Wrong. The way this erroneous headline escaped into the public domain, and came to encapsulate this week's wintry weather, provides a cautionary tale for all journalists and politicians.
The original statistic was released on Monday morning, and it said, correctly, that central London had experienced its heaviest snowfall since February 1991. About six inches lay in the London parks that day, and rather more than 12 inches in northwest Kent, north Surrey, and the south London suburbs. During the '91 snowfall a foot of snow fell even in central London.
Watching the BBC news channel during the day the headline gradually morphed from "the heaviest snow in central London for 18 years" to "the heaviest snow in parts of the South for 18 years" to "the heaviest snow in some parts of the country for 18 years", and by late-afternoon we finally ended up with "Britain's worst snowstorm for 18 years".
This would be a relatively trivial point if that were the end of the matter. But it is not. On 'Question Time' on Thursday Geoff Hoon, the Transport Secretary, excused local authorities in all parts of the country for their inability to deal effectively with the snow by saying that they cannot be expected to cope with events which happen only once every 18 years. The other panellists seemed meekly to accept
the statistic.
Once again ˜ as with the floods in summer 2007, the severe gale in January 2007, the pre-christmas fog in 2006, and countless other events ˜ incorrect facts and figures are used by those whose job it is to maintain the nation's infrastructure to evade responsibility for their failure. Only in London, northwest Kent and north Surrey was
such an evasion justfied.
One wonders how on earth we would all manage now if we were faced with a re-run of a winter like 1963 or 1947."
Saturday, 7 February 2009
Jamie's brain food
“An independent study shows the performance of 11year-old pupils eating Oliver’s meals improved by up to 8% in science and as much as 6% in English.”
Roger Waite, “Jamie’s food fuels pupils’ brain power”, Sunday Times, February 1, 2009
First of all, the report, Belot, M., and James, J., (2009) “Healthy School Meals and Educational Outcomes” (ISER) actually says that the number of pupils attaining level 4 increased “by 3 to 6 percentage points in English, and the percentage of pupils reaching level 5 (improved) by 3 to 8 percentage points in Science” (my italics). Admittedly, “up to” is correct, in the same way that “as little as 3 per cent” would have been equally as true. But whether 3, 6 or 8 per cent, an improvement is an improvement, right?
Well, first of all, the study which primarily compares improvements in English uses 7 other Greenwich schools as controls. These other schools have a “substantially lower… percentage of pupils speaking English as their first language.”
The researchers are no innocents here. This report is straining to prove the merits of Jamie Oliver’s campaign - the very thing that they purport to be independently researching. Even the opening pages, the authors can’t help their hyperbole: “healthy meals were being put alongside the original junk food” (my italics).
Incredibly, after noting the substantial improvements in the school serving Jamie Oliver’s dinners, they make the startling claim: “So far we have included all pupils in the analysis. However, only part of them has been truly treated, those who actually eat school meals. We do not have individual information about who is eating school meals and who is not.” Don't worry the "modelling" will even out that minor point, eh!
The possibility of the placebo effect seems to undermine any remaining credibility in this report. The authors say: “some of the treated schools were explicitly mentioned in (Jamie Oliver's TV) program, such that one could expect that for those schools the 'placebo-effect' could be stronger than others. We have extended the empirical analysis to allow for this possibility… (and there is) no way researchers can be sure that the effect they estimate is truly due to the change in policy rather than a placebo effect.”
I could go on, but the paper is full of provisos, and relies on “modelling” more than facts. A fair assessment of the research is that it is interesting, but not substantive. However, lo and behold, it was press released by the department and regurgitated by an unquestioning press.
The fact that this stuff can justify an additional £1.2 million expenditure by September 2007 alone, is tragic. Arguing for better school dinners should be enough. Trying to justify better school dinners on the basis of the mystical educational benefits of Jamie's Oliver's brain food, should be as unnecessary as it is impossible.
Roger Waite, “Jamie’s food fuels pupils’ brain power”, Sunday Times, February 1, 2009
First of all, the report, Belot, M., and James, J., (2009) “Healthy School Meals and Educational Outcomes” (ISER) actually says that the number of pupils attaining level 4 increased “by 3 to 6 percentage points in English, and the percentage of pupils reaching level 5 (improved) by 3 to 8 percentage points in Science” (my italics). Admittedly, “up to” is correct, in the same way that “as little as 3 per cent” would have been equally as true. But whether 3, 6 or 8 per cent, an improvement is an improvement, right?
Well, first of all, the study which primarily compares improvements in English uses 7 other Greenwich schools as controls. These other schools have a “substantially lower… percentage of pupils speaking English as their first language.”
The researchers are no innocents here. This report is straining to prove the merits of Jamie Oliver’s campaign - the very thing that they purport to be independently researching. Even the opening pages, the authors can’t help their hyperbole: “healthy meals were being put alongside the original junk food” (my italics).
Incredibly, after noting the substantial improvements in the school serving Jamie Oliver’s dinners, they make the startling claim: “So far we have included all pupils in the analysis. However, only part of them has been truly treated, those who actually eat school meals. We do not have individual information about who is eating school meals and who is not.” Don't worry the "modelling" will even out that minor point, eh!
The possibility of the placebo effect seems to undermine any remaining credibility in this report. The authors say: “some of the treated schools were explicitly mentioned in (Jamie Oliver's TV) program, such that one could expect that for those schools the 'placebo-effect' could be stronger than others. We have extended the empirical analysis to allow for this possibility… (and there is) no way researchers can be sure that the effect they estimate is truly due to the change in policy rather than a placebo effect.”
I could go on, but the paper is full of provisos, and relies on “modelling” more than facts. A fair assessment of the research is that it is interesting, but not substantive. However, lo and behold, it was press released by the department and regurgitated by an unquestioning press.
The fact that this stuff can justify an additional £1.2 million expenditure by September 2007 alone, is tragic. Arguing for better school dinners should be enough. Trying to justify better school dinners on the basis of the mystical educational benefits of Jamie's Oliver's brain food, should be as unnecessary as it is impossible.
Times2
Here's an interesting example of bad interpretations of science. I'm not sure whether of the concluding sentence is ironic, or not. But the report cited in this article is McPherson, K., Marsh, T. & Brown, M (2007), “Tackling Obesities: Future Choices – Modelling Future Trends in Obesity and the Impact on Health - 2nd Edition” (Foresight). While this blog will try to expose bad interpretations of reasonable research, this report has been produced by the UK Government’s Foresight Programme run by the Government Office for Science under the direction of the Chief Scientific Adviser to HM Government. In the next breath it states that “The views… are independent of Government”:
"Bad statistics Easy? Fat chance", Tom Whipple, Times2, February 6th 2009
"Fact: by 2050, 90 per cent of Britons will be obese. But fear not; by 2100, everyone will be anorexic. As well as being obese. Predicting the future is tricky. But when Alan Johnson, the Health Secretary, launched the Change4Life anti-obesity campaign, he felt able to give a killer angle: if nothing is done, by 2050 nine adults out of ten will be overweight.
His figures are based on the idea that obesity will keep on rising over the next 42 years as it has in the past 14. There is a f law in this plan, of which the authors of his study were aware: “Fitting straight lines to this data presents a problem as these would show some groups passing 100 per cent, which could not occur in real life.” So, if trends continued, more people would be obese than existed.
Instead they used: “A simple and convenient set of slowly varying, monotonic functions that are asymptotic to 0 and 1.” In other words, they fiddled it. This had a peripheral benefit: the calculations look very sciencey, with things such as p(t) = ½[1 + tanh(a+bt)].
But what about anorexics? Between 2002 and 2007 the number of hospital beds used in treating eating disorders increased by two thirds. If that rate continues, in 90 years everyone in the UK (including fat people) will be in hospital with an eating disorder. This is what we know: people are getting fatter; being fat is bad. We don’t know what will happen 42 years from now.
To get technical, why choose to make your trend asymptotic to 100 per cent of the population? Why not to 99 per cent, so excluding PE teachers and athletes? Why not to 85 per cent, in case 15 per cent of the population are genetically incapable of being fat? Perhaps it is worth making up numbers if it scares some fatties into slimming."
"Bad statistics Easy? Fat chance", Tom Whipple, Times2, February 6th 2009
"Fact: by 2050, 90 per cent of Britons will be obese. But fear not; by 2100, everyone will be anorexic. As well as being obese. Predicting the future is tricky. But when Alan Johnson, the Health Secretary, launched the Change4Life anti-obesity campaign, he felt able to give a killer angle: if nothing is done, by 2050 nine adults out of ten will be overweight.
His figures are based on the idea that obesity will keep on rising over the next 42 years as it has in the past 14. There is a f law in this plan, of which the authors of his study were aware: “Fitting straight lines to this data presents a problem as these would show some groups passing 100 per cent, which could not occur in real life.” So, if trends continued, more people would be obese than existed.
Instead they used: “A simple and convenient set of slowly varying, monotonic functions that are asymptotic to 0 and 1.” In other words, they fiddled it. This had a peripheral benefit: the calculations look very sciencey, with things such as p(t) = ½[1 + tanh(a+bt)].
But what about anorexics? Between 2002 and 2007 the number of hospital beds used in treating eating disorders increased by two thirds. If that rate continues, in 90 years everyone in the UK (including fat people) will be in hospital with an eating disorder. This is what we know: people are getting fatter; being fat is bad. We don’t know what will happen 42 years from now.
To get technical, why choose to make your trend asymptotic to 100 per cent of the population? Why not to 99 per cent, so excluding PE teachers and athletes? Why not to 85 per cent, in case 15 per cent of the population are genetically incapable of being fat? Perhaps it is worth making up numbers if it scares some fatties into slimming."
Tuesday, 3 February 2009
New Paint = Better Performance
"Painting classrooms raises results" says BBC News/ Education on 28 September 2005.
The notion that a lick of paint can do as much for school exam performance as better facilities or more and better teachers, has had the understandable effect of schools opting for the cheaper option.
Even though there is a great deal of inconclusive literature on the psychological effects of colour, etc, on behaviour, the main paper used to justify this claim is a 2005 study by Higgins, S., Hall, E., Wall, K., Woolner, P., McCaughey, C., (2005) "The Impact of School Environments: A literature review", Newcastle/ Design Council.
It states that "some physical elements in the classroom improve comfort, well-being and probably attitude - and so, perhaps, improve acheivement".
It also warns that: "It is important, therefore to beware of 'architectural determinism'."
Whatever you think of the actual survey, the conclusions are being used in precisely the way that the authors warned... but that hasn't stopped the performance-enhancing qualities of coloured paint becoming an orthodoxy.
The notion that a lick of paint can do as much for school exam performance as better facilities or more and better teachers, has had the understandable effect of schools opting for the cheaper option.
Even though there is a great deal of inconclusive literature on the psychological effects of colour, etc, on behaviour, the main paper used to justify this claim is a 2005 study by Higgins, S., Hall, E., Wall, K., Woolner, P., McCaughey, C., (2005) "The Impact of School Environments: A literature review", Newcastle/ Design Council.
It states that "some physical elements in the classroom improve comfort, well-being and probably attitude - and so, perhaps, improve acheivement".
It also warns that: "It is important, therefore to beware of 'architectural determinism'."
Whatever you think of the actual survey, the conclusions are being used in precisely the way that the authors warned... but that hasn't stopped the performance-enhancing qualities of coloured paint becoming an orthodoxy.
The bigger the window, the more children learn
"Fenestration was carefuly developed for all classrooms to maximise daylight which has been proven to improve the speed of learning", BDP, "BDP wins Lighting Design Award', Press Release, 10 APril 2008.
Think about how stupid this assertion is. Maximising daylight "has been proven" to "improve the speed of learning". Notwithstanding the fact that dirt poor developing countries - where children are taught in dingy huts with meagre light provision have some of the most dedicated students - there is no mystical relationship between windows and the ability to learn.
Unsurprisingly, when you dig out the research, nothing of the sort has been "proven".
This mainstream assertion is openly credited as coming from Heschong, L., "Daylighting in Schools - An Investigation into the Relationship between Daylighting and Human Performance", Pacific Gas & Electric Company, 1999. In it you'll find the statement that "this kind of observational study cannot determine a causal relationship". It also goes on to hint that teaching methods and educational standards have a great deal to do with the results! This aspect is seldom revealed.
Think about how stupid this assertion is. Maximising daylight "has been proven" to "improve the speed of learning". Notwithstanding the fact that dirt poor developing countries - where children are taught in dingy huts with meagre light provision have some of the most dedicated students - there is no mystical relationship between windows and the ability to learn.
Unsurprisingly, when you dig out the research, nothing of the sort has been "proven".
This mainstream assertion is openly credited as coming from Heschong, L., "Daylighting in Schools - An Investigation into the Relationship between Daylighting and Human Performance", Pacific Gas & Electric Company, 1999. In it you'll find the statement that "this kind of observational study cannot determine a causal relationship". It also goes on to hint that teaching methods and educational standards have a great deal to do with the results! This aspect is seldom revealed.
Green spaces help you live longer
"Good neighbourhood green spaces promote longer life expectancy for local people".
so says Stuart Lipton. ex-chair, Commission for Architecture and the Built Environment (CABE) in "The Value of Public Space" (2003). The same report says that: "a walk in the park to aid patients’ health... has been proven to reduce the risk of a heart attack by 50 per cent".
This nonsensical conclusion is arbitrarily culled from Hakim, AA et al (1999) "Efects of walking on coronary heart disease in elderly men: the Honolulu Heart Program" Circulation, Vol 100, pp9-30. This was a study of 2500 men of Japanese descent living in Honolulu and between the ages of 71-93. Their walking habits were assessed against average "life expectancy". The report actually concludes with a social policy objective that "even if walking does not have an independent effect on coronary heart disease, its potential effect through unknown pathways makes it worthy of promotion". Health trrends in Hawaii however, simply notes that: "Gains in life expectancy every 10 years mirror major developments in public health and medicine... the largest life expectancy increase, of almost four years, ocurred between 1970 and 1980".
At its most basic level, to draw a parallel between octogenarians strolling for unspecified amounts of time, over unknown terrain, in Honolulu ought not be turned into a generalist statement that a walk in the park reduces the risk of heart attacks by 50 per cent. What could this statement possibly mean? Unfortunately, it is now regularly regurgitated in urban design conferences, to justify spending on parks and green spaces. Arguing for a park, without the "research shows" label, it seems, would be too difficult.
so says Stuart Lipton. ex-chair, Commission for Architecture and the Built Environment (CABE) in "The Value of Public Space" (2003). The same report says that: "a walk in the park to aid patients’ health... has been proven to reduce the risk of a heart attack by 50 per cent".
This nonsensical conclusion is arbitrarily culled from Hakim, AA et al (1999) "Efects of walking on coronary heart disease in elderly men: the Honolulu Heart Program" Circulation, Vol 100, pp9-30. This was a study of 2500 men of Japanese descent living in Honolulu and between the ages of 71-93. Their walking habits were assessed against average "life expectancy". The report actually concludes with a social policy objective that "even if walking does not have an independent effect on coronary heart disease, its potential effect through unknown pathways makes it worthy of promotion". Health trrends in Hawaii however, simply notes that: "Gains in life expectancy every 10 years mirror major developments in public health and medicine... the largest life expectancy increase, of almost four years, ocurred between 1970 and 1980".
At its most basic level, to draw a parallel between octogenarians strolling for unspecified amounts of time, over unknown terrain, in Honolulu ought not be turned into a generalist statement that a walk in the park reduces the risk of heart attacks by 50 per cent. What could this statement possibly mean? Unfortunately, it is now regularly regurgitated in urban design conferences, to justify spending on parks and green spaces. Arguing for a park, without the "research shows" label, it seems, would be too difficult.
Subscribe to:
Posts (Atom)