The Department for Education and Skills has replied to our "anonymous human FOIA proxy") request about the OECD PISA study, made at the suggestion of someone posting a suggestion to this website.
This concerns the OECD PISA study of the educational standards achieved by 15 year olds in 51 countries around the world. Astonishingly, the DfES, through its contractor the Office for national Statistics, somehow managed not to sign up enough schools and pupils in England for the test results to be considered statistically valid enough for international comparision, the whole point of the exercise, and something which has been achieved in previous years.
This is despite increasing financial incentives from £200, to £500 to £1000 in cash to each participating school.
This all sheds an interesting light on the reign of Charles Clarke, who was the Education Secretary at the time, before being appointed Home Secretary a couple of months ago.
As an FOIA request, this one seems to have been successful, with some previously unreleased documents being made available, carefully redacted to remove the personal names of DfES and OECD officials, which is perfectly acceptable, whilst showing the sequence of events.
For those of you interested in comparative educational standards, or wishing to question the Government on school standards, this FOIA reest could prove useful for further probes e.g.
- Scotland and Northern Ireland did manage to provide a statistically valid sample to the OECD. What about Wales ?
- If the ONS cannot persuade enough schools to participate, should the contract be given to someone else ?
Letter from the DfES, via email, 11th February 2005, i.e. after only 15 working days:
"department foreducation and skills Sanctuary Buildings
Great Smith Street
Westminster
London
SW1P 3BTtel: 020 7925 5000
fax: 020 7925 6000
info@dfes.gsi.gov.uk
Direct line: 020 7925 nnnn
AAA.BBB@dfes.gsi.gov.ukOur Ref: 2005/0003903
10 February 2005
address:
Dear Mr. XXX
Freedom of Information Act Request - PISA
I am writing to confirm that the Department has now completed its search for the information you requested on 23 Janauary 2005. I wish to advisse you that some of theinformation cannot be disclosed and has been redacted because of section 40 personal information. This is an absolute exemption and needs no public interest test. I enclose a copy of the information which can be enclosed.
having studied your request, I thinkit would be useful firstly to clarify some aspects of the conduct of PISA, and the participation of England in the study.
For the first part of your request, regarding information made available to OECD for PISA 2003, please note that the day to day conduct of the study in England (eg the administration of the PISA tests and questionnaires in schools, the collection of data from these instruments, and the provision of those data to the OECD PISA consortium) was carried out by the Office for National Statistics, the Department's contractor. if your interest is in the information and data supplied by England for the PISA study you are likely therefore to find it more fruitful to address your enquiryy to ONS, although iin doing so, you should be aware that that schools and pupils participatein such studies on the understanding that they will remain anonymous. The appropriate conatct in ONS would be CCC DDD (tel: 020 nnnn nnnn, email CCC.DDD@ons.gsi.gov.uk)
In addressing the second part of your request, I should clarify that no decision was ever made to pull England out of PISA 2003, thaat England participated fully in the conduct of the study, and that all the data for England and the UK are in the public domain. What I think you probably refer to are the technical concerns the OECD has about the England and UK sample. Schools participate in PISA on an entirely voluntary basis and these concerns arose as we were unable to persuade enough schools and pupils to take part in the study and hence, to fulfill the OECD's strict criteria for response rates.
Having considered the sample of English schools and pupils ONS was able to secure in PISA 2003, the OECD took the view (as they did for the Netherlands in PISA 2002) that the sample waspotentially biased and that the UK data could not be considered suitable for international comparison. In consequence, the data for England and the UK were treated differently to that of other countries. You may find it useful to see the attached OECD paper to the PISAA Governing Board(1), which sets out their proposed text for Annes A3 of the PISA 2003 report and their detailed reccomendations for the treatment of UK data in the light of their technical concerns.
The treatment of UK data in the OECD PISA 2003 report (available from www.pisa.oecd.org) is consistent eith OECD's reccommendations. Moreover, aall the data for the UK and England, including those data and analyses considered not internationally comparable by OECD, are publicly available at www.pisa.oecd.org. Should you wish to use these data to make an assessment of the relative performance of the UK and English educational system, you should of course bear in mind OECD's warnings about the interpretability of the data, as set out in the attached note and Annex A3 of the report.
I hope the information offered here has provided useful clarification on your request. To give you a fuller background to the issues with response rates and the treatment of our data in PISA 2003, I also attach an internal note on the timeline of events. I hope that if the material provided does not meet your requirements it will allow you to submit a further, refined request to which I would be happy to respond.
I amalso obliged to tell you that if you are unhappy with the way the Department has handled your your request, you may ask for an internal review. You should contact me if you wish to complain.
if you are not content with the outcome of the internal review, you have the right to apply directly to the Information Commissioner for a decision.
If you have any queries about this letter, please contact me. Please remember to quote the reference number above in any future communications.
Yours sincerely
AAA BBB
Policy & Performance Division(1) Note that this document was not previously in the public domain, but that the OECD have agreed to us releasing it under FOI"
Attached documents:
- The OECD Annex A3 (170 Kb .pdf)
- The DfES Timeline of events (715 Kb .pdf)
http://www.publications.parliament.uk/pa/cm200506/cmhansrd/cm050526/text/50526w15.htm#column_208
"PISA Survey
Mr. Gibb: To ask the Secretary of State for Education and Skills which organisation will be conducting the next round of the PISA survey. [582]
Jacqui Smith: Following a competitive tendering exercise in 2004, the National Foundation for Educational Research (NFER) was appointed to administer PISA 2006 in England, Wales and Northern Ireland. The Scottish Executive conducted a separate exercise and has appointed The SCRE Centre, University of Glasgow to run the study in Scotland.
26 May 2005 : Column 209W"