Primary care: national monitoring and evaluation strategy

Methods and Core Evaluation Questions

Quantitative data and statistics-based indicators can tell us some of what is happening, where it is happening, and by whom and to whom. They usually cannot tell us about: why and how changes and outcomes occur; explain variations and the unexpected or unintended consequences of policies; reveal what people think and feel; or explain resource implications and trade-offs. On its own, quantitative data seldom allows for reflection on policies or the identification of options. For that, the triangulation of evidence and mixed methods primary research are generally required.

To provide decision makers with the best available and most appropriate local, national and international evidence, our approach encourages the use and triangulation of varied methods and evidence sources to answer evaluation questions. These could include evidence reviews (including international studies and policies), evaluations of policy initiatives, routine data, qualitative research, “middle ground research”, primary and secondary research, economic studies, and public engagement methods.^[30] As noted above, there will be an annual workplan to deliver this Strategy through specific projects and initiatives. This will be underpinned by detailed consideration of the best and most cost-effective methods and sources for addressing evaluation priorities. 

Questions for primary care at the national level 

We have suggested some core questions to shape national research and evaluation on primary care policy for both accountability and learning. We hope that others will also use these questions and the Outcomes Framework as useful tools to shape and guide (and facilitate greater comparability across) research and evaluation activity.

• How are major national commitments being implemented? Are they achieving their objectives, and how?
• To what extent are we making progress towards achieving each of the six primary care outcomes, and how? 
• What impacts have national programmes and investment had on sustainability and productivity in primary care, including delivering the “triple aim” of better health, better care, better value?
• What impacts have national programmes and investment had on people who use services and what matters to them?
• To what extent have new models of primary care contributed to the 2020 Vision of supporting people to remain at or near home where possible?
• What impacts have national programmes and investment had on supporting the development of extended MDTs, and why?
• What factors have supported or hindered the effectiveness of new models of care (including local contextual variation, external factors, unforeseen events)?
• Over time, do primary care policies and structures remain fit for purpose, to meet local and national needs? Are they supported by the best available evidence?
• What impacts has primary care reform had on other parts of the wider health and social care system? And wider system reform on primary care?

Questions for programmes and projects^[31]

It may also be useful to consider a set of core questions to use alongside any research questions which are specific to a project or programme, at different levels or scales. For example:

• What was the need and intended outcomes for the change?
• How was the activity to be tested chosen or new approach developed?
• How were the projects and policies implemented (including resources required); and was this as planned? Where relevant, was there fidelity to the model being tested?
• What external factors supported or hindered implementation?
• What are the outcomes for both people who use services and for the workforce (e.g. quality of experience, wellbeing, perception)?
• What are the outcomes for the system (e.g. on access, demand, sustainability, efficiency and productivity, (cost-)effectiveness, safety and quality)?
• To what extent are short and medium-term outcomes achieved and attributable to the new policy or way of working?
• Were there any unintended consequences (positive or negative)?
• What are the ‘active ingredients’ of the project or programme? Which aspects, if replicated elsewhere, can be expected to give similar results and what contextual factors are needed for success? 
• What impacts has the programme or project had on other parts of the wider health and social care system?