Strategic Forum for Research in Education

What quality assurance and accountability procedures are in place for educational research and development?

There is much good quality research in the area of children and education being undertaken across the four countries of the UK. However it is important to note that while the overall quality is good, there is considerable variation across the UK countries.

The quality of education research across the UK higher education sector has risen significantly in recent years as demonstrated by the results of the recent Research Assessment Exercise (RAE). In the last RAE (2001) only 13 universities were awarded grades from ‘international excellence’ whereas in 2008 the work of 39 institutions was judged to be ‘recognised internationally’ or better. However it is important to note that while overall quality is improving, there is considerable variation across the countries of the UK and there remains much work to be done. The report of the Education RAE panel describes the outcome in more detail.

Whilst a central location for much academic research work in education takes place in university departments, there are considerable amounts of work undertaken by government analysts, educators, research organisations, private consultancies, charities and think tanks.

Does quality matter?
Quality of research matters for different stakeholders for often different (though overlapping) reasons. In her summary of the discussions at Forum I, Dr Alis Oancea highlights three motivations for assessing research quality and why quality: high quality research is needed to have a positive impact on practice and policy making through direct impact but also through supporting a research culture; assessments of quality are also important for individuals; quality of research is also vital for the health and development of the field, building and accumulating knowledge and understanding (see p. 19 – 20 of the report of the first Forum).

What criteria matter?
Judging the quality of education research is itself a challenge, and no single set of criteria or indicators can capture the multiple purposes for which research and enquiry activities are carried out. The way forward was felt to be in recognising such different purposes and in managing the consequential tensions they create. Often the importance of different criteria depends on which side of the knowledge management system you are on – i.e. a user or a producer of research. Impact (or significant) is an important element of quality particularly for users such as policy makers and practitioners.

Research undertaken in the field of social policy analysed researchers’ views of what was important in terms of assessing quality and found that there was not a consensus with regard to what criteria were important (Becker, Bryman & Sempik, 2006). The authors conclude that views on the importance of different criteria depend on the evaluator’s view of what research is for.  They define five groups: research process orientation; policy orientation; theoretical orientation; service user orientation; and academic prestige orientation.

How is quality assessed?
Quality is generally assessed through using a peer review process (see MacNab and Thomas, 2007). Assessments of quality for publication in academic journals rely on a formal peer review process and internal processes within research organisation often use expert peer review as a means of quality assessment. Peer Review: The Challenges for the Humanities and Social Sciences – published by The British Academy – reviews the practice of peer review and finds, whilst peer review is not perfect, there is no better way of assessing quality and that many of the criticisms of peer review are concerned with the practice rather than the concept. The report concludes that focus should be on improving the process through training and developing practice. Peer review takes place ex ante in the form of reviewing of proposals for funding as well as ex post through review for publication. Commissioners of research, journal editors, RAE panel members and reviewing bodies of research councils are the gate-keepers of quality in this respect.
Whilst the RAE (mentioned above) process assesses research undertaken within university departments, there is little known above the quality assessment processes and criteria used in other organisations (including government departments).

Outstanding issues

• How valuable in enhancing quality is the co-production of education research involving policy makers, practitioners and researchers?
• Is it important to engage users of research with quality assessment? If so, how can this be achieved?
• How can assessment processes better support quality enhancement?
• How can discussions regarding quality between stakeholders improve quality?
• How accountable should commissioners of research (including government departments) be for the quality the research they commission and, for government in particular, for the research used to inform policy decisions?

 

Further references

• Towne, L., Wise, L. L., & Winters, T. M. (Eds.). (2005). Advancing Scientific Research in Education. Washington, DC: The National Academies Press.
• Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: The National Academies Press.
• Wilson, A. (2008). Punching our weight: the humanities and social science in public policy making – A British Academy Report. London: British Academy.