The Huddersfield Policy and Prevention Analysis Site
Professor Ken Pease (2002-2004)
Professor Alex Hirschfield (2005-2007)
Institution: University of Huddersfield
Through the Huddersfield site, the Network aimed to analyse approaches to policy and prevention. The Huddersfield sit has produced two reports as a culmination of its exploration of current and potential methods for translating research findings into guidance for policy and practice.
- A framework for deriving policy implications from research. This report investigates the current state of evidence-based policy making and addresses a number of questions about the most effective ways for evidence to inform policy more generally. This report can be found here.
- Pathways into policy: A study of the relationship between research, policy and government. This report considers effective methods for generating policy implications from research like that undertaken at SCoPiC research sites, while taking into account methodological and ethical constraints. This report can be found here.
Key goals of Huddersfield research were
- To study the relationship between policy
- To make recommendations designed to enhance the influence of research on policy
To do so the team
- Compared approaches to policy making in criminal justice versus healthcare
- Reviewed existing literature
- Interviewed key policy makers from the Home Office, the Youth Justice Board, the Department of Health, NICE, ACPO and HM Prison Service, among other government agencies
back to top
- Deriving policy implications from research
Translating research findings into effective policy and policy into effective practice involves a number of processes, including
- Analysis for policy – Research into the nature and manifestation of a phenomenon/problem (for example, offending) including exploration of possible causal mechanisms in order to inform ameliorative policies
- Policy analysis – An examination of how different policies likely to influence a given outcome (for example, offending) related to each other in terms of their aims, tactics, resources, timing, location and intended beneficiaries
- Policy design – The formulation of a policy (i.e., objectives and tactics) for tackling an identified problem with some initial assessment of the feasibility of implementation and/or likely cost effectiveness
- Policy delivery – All aspects of policy implementation including forward planning, management, data sharing, expenditure, timetable, targeting of resources and performance management
- Policy evaluation – An assessment of the impact and effectiveness of a policy in achieving state outcomes; this may be undertaken concurrently with policy implementation or retrospectively (ex post evaluation)
For the first of their reports, the Huddersfield Policy and Prevention Analysis team focused on the first of these: analysis for policy. The government has expressed a commitment to evidence-based practice, which raises the question of how the results of empirical research can best be fed into the policy process. In an attempt to answer that question, the Huddersfield team developed a framework for deriving policy implications from research findings.
The first stage of the process involves assessing the rigour of the research, e.g., the generalisability of the results and the extent to which the variables used represent accurate measures of the theoretical constructs under examination. Next, the amenability of causal variables to change through policy intervention is assessed to determine the feasibility of targeting those variables. Finally, the ethics of changing those variables in assessed, before various policies are considered.
Key features of the model:
- Is the research internally valid? Consider:
- Defensibility and “fitness for purpose”
- Data collection and analysis
- Reflexivity and objectivity
- Is the research externally valid? Consider:
- Sampling method
- Context of the research
- Are the key constructs valid? Consider:
- Are there any identifiable policy implications?
- Can the casual variables be manipulated?
- Are they based on data sources not normally available for the population?
- Would the cost/time involved be prohibitive?
- Is there any information on the likely size of benefit?
- Would the changes infringe personal freedom/legal rules?
- Would the changes conflict with the values that underpin policy making?
- Are there any likely unintended consequences?
- Practical Considerations
- What are the implied policy changes?
- At what level would they operate?
- Who is the target of the policy?
- Who would be responsible for funding/delivery?
- Over what timescale would they be needed?
Further testing and use of the framework will be necessary before deciding whether it process a useful and practical tool. However, the Huddersfield team hopes that such a model may encourage researchers to be more rigorous in reporting the aspects of their research which may be of most relevance and interest to policy makers and that, as a consequence, policy makers will be inclined to place more weight upon the findings of criminological research.
back to top
- Relationships between research and policy
In recent years the government has called for a more ration, evidence informed policy making process. The concept of evidence-based policy raises a number of issues, including
- What is evidence?
- What role should evidence have in the policy process?
- What are the obstacles to using evidence?
- How can these obstacles be overcome?
The Huddersfield team explored these questions through a combination of reviewing the literature and carrying out in-depth interviews with policy makers in criminal justice and, comparatively, healthcare.
Policy-making is a process of making decisions about what to do (or not do) and usually involves identifying alternatives and choosing among them on the basis of factors which policy makers consider relevant or important. Evidence is less easy to define, but can be taken to mean information which helps to support or challenge a policy. Whilst this would often take the form of empirical, largely quantitative, academic research, policy makers recognised the value of other forms of evidence, such as public and stakeholder opinion and qualitative research. Interviews with policy makers revealed they considered evidence to be a key criterion. When asked what types of evidence they found most convincing, policy makers tended to talk about ‘fitness for purpose’ rather than hierarchies of evidence; in other words the selection of evidence depended on what was the appropriate methodology for the question at hand. After evidence, policy makers in criminal justice tended to be concerned about cost, and those in healthcare, legal constraints.
The descriptions of policy making in the various organisations studied revealed differences in approach to and the weight assigned to evidenc; in some organizations (for example, the Home Office) the search for evidence was less systematic and the influence of political and other considerations higher, such that the process could best be described as ‘evidence-aware’. Other organizations (for example, NICE) demonstrated a structured evidence based approach to producing guidance (policy).
In general, evidence appeared to be less central to policy making in criminal justice than health, and only in the latter sphere were there concrete examples of bodies established to promote evidence based healthcare. Recent examples of criminal justice policy making, including large scale crime reduction programmes and interventions funded by the YJB, suggest that criminal justice policies are influenced primarily by factors other than evidence, such as political expediency.
Funding for research and development in criminal justice was found to be disproportionately small in relation to the scale of the problem:
Government expenditure on crime research
RDS £12 million
CRP, YJB, BCS £25 million
ESRC & Others £13 million
Total Estimate £50 million
Government expenditure of health research
DoH £753 million
MRC £547 million
Total Estimate £1300 million
Government expenditure on the problem of crime
CJS Administration £20 billion
Costs of crime £48 billion
Total Estimate £68 billion
Research funding as a % of the cost of crime: 0.07%
Funding was also directed largely at areas of research unrelated to the causes of crime or means of its prevention. These problems need be addressed before any substantial progress can be made in terms of delivering evidence-informed criminal justice policy. The quality of research, particularly that aimed at answering the question ‘what works’, is often poor, and while some of the blame may lie with under-funding, researchers could do more to address issues of validity. At the same time, control over the direction of the research budget in criminal justice is overly political; there is at present, no means for ensuring that knowledge about what does (or does not work) gets translated into authoritative guidance for practitioners.
The relationship between research and policy stands or falls on the relationship between those most intimately involved. What is needed is not just a dialogue between researchers and policy makers, but they need to speak the same language. Too often, potentially useful research findings are lost to the policy process because they are narrowly disseminated to the academic community or their policy implications are not drawn out.
Proposed solutions to the obstacles to evidence-based practice:
- Achieving a match between the level of resources and the scale of the problem is just as important when funding research, as it is when delivering policy.
- Even without a significant increase in overall funding, the value of criminological research to policy makers could be increased by focusing less on management statistics or ‘user satisfaction’ surveys, and more on primary research which aims to extend our knowledge base in this complex area.
- The single most important difference between the criminal justice and health policy making system was the presence of an independent body (NICE) which delivered evidence-based guidance directly to practitioners.
- Researchers need to become better at anticipating policy makers’ needs for research, and at anticipating objections to the conclusions of their research. A greater awareness of the policy process will also enable researchers to take account of the many other pressures policy makers face and which influence their decisions.
- Similarly, knowing, who, what and when, the researcher can then target relevant policy makers with relevant research when they are more likely to be responsive, bearing in mind policy makers’ key requirements of research that it be brief and suggest practical implications.
- For their part, funding bodies should aim to anticipate the needs of policy makers, so that research which is of relevance to the policy agenda can be delivered when it is needed. Practical support for researchers might include provision of dedicated time and resources within research budgets for this purpose, and the requirement that a dissemination strategy form part of the research design.
- The ‘policy-friendliness’ of research can be increased through use of appendices or annexes, (to maintain the flow of text in the main body), case studies, examples of best practice, executive summaries and a brief statement of research implications.
- It is not enough to publish and hope one’s ideas will percolate; success involves presentation, networks, timeliness, language and ‘being in the right place at the right time’.
back to top
Interviews with policy makers revealed that they considered evidence to be a key criterion. When asked what types of evidence they found most convincing, policy makers tended to talk about ‘fitness for purpose’ rather than hierarchies of evidence; in other words the selection of evidence depended on what was the appropriate methodology for the question at hand.
When the research team looked more generally at how government departments described the policy making process and what structure they had put in place, evidence appeared to be less central to policy making in criminal justice than in health. This led on to an investigation of what might be the causes of the less than perfect relationship between research and policy. Some possible causes include
The fact that funding for research in criminal justice was disproportionately small in relation to the scale of the problem; funding for criminological research represents just 0.1% of the cost of crime to the economy
- The fact that much of the research which is funded is not concerned with uncovering the causes of crime
- The fact that the quality of research was often poor, particularly that of research aimed at answering the question ‘what works’
- The fact that control over the direction of the research budget in criminal justice was overly political, with no means for ensuring that knowledge about what works gets translated into guidance for practitioners
Issues which policy makers and researchers might wish to consider, if the relationship between them is to become more fruitful include:
- The external environment. External influences affect how people think and behave. Who are the key external actors (researchers, policy makers, stakeholders, etc.); what is their agenda; what is their influence in the political context?
- The political context. The influence of evidence depends in part on a number of factors, including: is there is political interest in change; what are ministers’ political priorities; how do ministers perceive the problem; is there public pressure for change; is their room to manoeuvre?
- The evidence. What evidence is available; how accessible it is; is there enough of it; is it rigorous and relevant; does it imply practical, achievable policy actions; are the concepts familiar or new; does it need repackaging?
- The links between researchers and policy makers. Evidence is impotent unless it can be brought to the attention of policy makers. Who are the key organizations and individuals; are there existing networks to us; could new networks be developed; do policy makers rely on limited source of information; what is the best way to penetrate their networks and transfer the information?
back to top
There is scope for further research into the area of criminal justice policy making. Just as many criminologists have come to recognise the importance of context in developing theories of involvement in and desistance from crime, so context is key to understanding the impact of evidence on policy.
back to top Back to Studies page