Impact assessment in governments: literature review

5. How is integration of impact assessments handled?

Section 2.2 noted that Ireland and Wales have 'integrated' policy impact assessment requirements. In Ireland, RIA includes an assessment of impacts on equality, the environment, and rural communities; and in Wales the wellbeing assessments cover include health, equality, the environment, and climate. But how does 'integration' occur? What happens if there are tensions within these 'integrated' assessments? And what happens if different types of assessment (e.g. RIA, HIA) of the same policy lead to different conclusions about the best direction for the policy?

5.1 What is integration?

'Integration' is not one unique state, but covers a continuum from full integration through to minimisation of impacts (Figure 5.1).

5.2 Integration within impact assessment processes

The literature identifies a wide range of impact assessment types that can be integrated, with some groupings of impact assessment being more obvious than others (Figure 5.2):

• Environmental + health (environmental risk)
• Health + equality (health inequalities)
• Environmental + health + equality (US 'environmental justice')
• Poverty + rural + 'equality groups' (who wins, who loses)
• Environmental + social (England and Wales 'sustainability appraisal')
• Within equality assessment, consideration of different groups (men/women, people with and without disabilities etc.)

Integrated assessments can reflect broader political priorities beyond the policy relevance of individual policies or regulations. This increases the relevance of the assessment, making it more tailored to political demands (OECD, 2011). However, both Jacob et al. (2011) and Adelle and Weiland (2012) found that, in practice, integrated impact assessments tended to focus more on regulatory burdens and economic competitiveness than on other dimensions such as equity and the environment.

There is little literature on the process of integrating impact assessments, and particularly none that link this with the hierarchy of integration shown at Figure 5.1. Johnston (2017) describes Irish attempts to integrate different types of assessment, which concluded that integration was too difficult in the short term:

"A particular consideration at the time of the early reviews [of poverty-proofing in Ireland] was how to develop a more integrated system of proofing, including, for example, poverty, gender and equality, and possibly rural and environmental issues as a more encompassing tool for policy proofing, rather than carrying out a number of parallel exercises. A number of groups and studies considered these issues… but found certain limitations to the development of such an integrated system… [It was] concluded that an integrated process of policy proofing should be developed when the existing parallel processes were fully operational, and should incorporate a screening mechanism and greater coordination of guidelines and advisory functions" (Johnston, 2017).

Kearns and Pursell (2011) also noted that key stakeholders in the environmental, health and local government divisions of the Irish public sector appeared to be working in parallel rather than collaboratively with respect to HIA.

The literature does identify factors that affect how well different forms of assessment can be integrated. In turn this gives an indication of what should be considered when considering how to integrate different forms of assessment into one process:

Data availability: Data may be more easily available for some forms of impact assessment than others. For instance, data on gender equality is comparatively easy to find and present. In contrast, disability data relies on self-disclosure and there is substantial variation in the range and nature of disabilities (OECD, 2019).

Quantification: Impact assessments typically focus on monetisable, or at least quantifiable impacts of policy proposals, and neglect those social and environmental aspects that are not monetisable or easily comparable (Turnpenny et al., 2009). Some RIA systems that use cost-benefit analysis aim to quantify/monetise as many impacts as possible, and to conclude with a single indicator (money). However Adelle and Weiland (2012) note that this does not allow trade-offs to be sufficiently acknowledged.

Single v. multiple indicators: Impact assessment is easier when fewer dimensions/indicators are involved. For instance, the OECD (2019) has suggested that moving from 'gender budgeting' to 'equality budgeting' in Ireland has meant moving from a single dimension (gender) to multiple dimensions (gender, poverty, people with disabilities, rural communities), making it more complex for departments to implement. However even the gender-as-one-dimension approach is countered by the National Women's Council of Ireland (2017), which perceives gender as only one facet of the much more complex problem of poverty, equality, fairness, social rights and economic resilience:

"Successful gender budgeting requires understanding of the dynamics of gender and inequality… [The International Monetary Foundation] has concluded that the issues of growth and equality can't be separated, rather there's a strong case for thinking about inequality and inability to sustain economic growth as two sides of the same coin".

Assessments involving long time scales (e.g. sustainability, future generations) are also difficult for policy-makers to handle (Nykvist and Nilssen, 2009).

There is also the 'detail paradox', which states that the power of each objective diminishes with the addition of other objectives: in other words, the more detailed the assessment is, the less significance, on average, is attached to each detail. Bondemark et al. (2020) note, for instance, that Swedish transport plans have so many goals and subgoals that cannot be summed to a single indicator, that this diminishes the governing power of each subgoal.

Mutual understanding of terminology and frames of analysis: Assessment terms must be explicitly stated and understood (WHO, 2008; Johnston, 2017). Verloo (2007) gives a Dutch example of government departments increasingly having 'gender blind policy frames', in contrast with the 'feminist frames' of equality officials and gender experts^[8]. EPA (2015) research found that Irish government and private industry stakeholders had at least four different interpretations of what health means in terms of the environment-planning interface, from health impacts being adequately assessed using only environmental limit values through to HIA being a tool to appraise both physical and social determinants of health. Sherry and Shortall (2019) similarly found that there was 'ambiguity, or lack of practicable understanding' of the key terms and concepts related to rural proofing. Their interviewees – staff from three categories of public bodies subject to the Rural Needs Act (Northern Ireland Government Departments, Local Government Districts and Non-Departmental Public Bodies) and key informants - expressed confusion regarding the relationship between equality, equity and rights.

When trying to integrate HIA into SEA in France, "From the very first exchanges between [working group] members, it became clear that work was required on finding definitions that suited everyone for the main concepts: Health, determinants of health, environmental health…" (Roué Le Gall et al., 2018). Molnar et al. (2016) refer to the need for a 'shared language', and suggest that this can only be achieved after time spent aligning conceptual understandings to bridge the sectoral cultural divides.

Intersectoral cooperation: The WHO (2018) suggests that lack of intersectoral cooperation poses one of the main barriers to HIA-SEA integration. This is due cultural differences between the sectors and the different 'languages' they use, different governance arrangements, and a lack of trust between the different stakeholders (Molnar et al., 2016; WHO, 2018)^[9]. Turnpenny et al. (2009) note that the 'silo' culture of policy making is still a significant constraint on integrated policy assessment. Roué Le Gall et al. (2018) also note that partnerships between regional health authorities and urban planning teams to integrate health issues into SEA requires a 'shared culture' to be established. Molnar et al. (2016) suggest that some of this shared culture can come from piggybacking the assessment aims onto pre-existing and well-understood principles like sustainable development.

Expertise: 'Integrated' assessment may be carried out by experts in one type of assessment, who may know little about the other types of assessment, but is this a problem? Dauvin (2005) argues that decision-makers and planners need to draw on those most knowledgeable when conducting environmental impact assessments. In contrast, EOHSP (2007) claims that HIA is neither difficult nor overly scientific, with common sense and persuasive powers being more important than health specialisations: "We should be persuading many people that they are capable of performing HIA, promoting confidence by helping them to acquire extra skills".

Focus on win-win: Pinto et al. (2015) and Molnar et al. (2016) note that policy goals that are framed as providing 'win-win' health and economic benefits have a high degree of buy-in^[10]. Similarly, gender budgeting gains more traction where it is reframed as being central to economic policy and inclusive growth (National Women's Council of Ireland, 2017). Molnar et al. (2016) suggest that non-health sector buy-in can also be achieved by health impact assessment experts working with policy-makers on their own agendas (e.g. improving educational outcomes for vulnerable students rather than improving healthy eating behaviour) rather than vice-versa. They developed a framework for achieving win-win solutions, which incorporates many of the additional points raised at Section 6.

5.3 Integration of impact assessment findings into policy

The types of impact assessment discussed in this report, with the exception of RIA, are all likely to go counter to short-term economic optimisation: they involve adding environmental and health safeguards, focusing on the longer term (climate change, wellbeing of future generations, sustainability) and rebalancing policy to take account of the needs of disadvantaged groups. Turnpenny et al. (2009) note that RIAs tend to be dominated by an economic growth paradigm which constrains the issues covered in assessments and focus on economic costs rather than environmental impacts. Molnar et al. (2016) note that, in Sweden, the business sector is particularly resistant to HIA awareness-raising "as there was no apparent profit to be made from intersectoral work".

Unsurprisingly, most of the examples in the literature are of impact assessments failing to affect policy (Figure 5.1):

• In the Netherlands, an HIA for a national approach to reducing obesity suggested analysing the effects of food advertisement restrictions and food labelling. However these topics were excluded from further consideration because the stakeholders devising the approach – who included food, drink, and catering industries - were committed to self-regulation and cooperation rather than legal measures. Indeed, the public health policy-makers themselves ended up by rejecting the HIA, arguing that it lacked rigour (Bekkers, 2007) = minimise impacts
• A study of HIA in Sweden, Quebec and South Australia (Pinto et al., 2015) found that health issues alone 'count for nothing' for ministries whose primary goal is economic. An interviewee within that study noted that when a policy has an impact on business outcomes, it creates 'incredible tension' between economic and social deliverables, for instance between increasing employment and improving health outcomes = minimise impacts
• An HIA was carried out at the instigation of a Traveller community in Ireland, who wanted to change the local authority's policy stance on Traveller accommodation. However, this led to little political action, and "the HIA was subsumed in a policy discourse marred with prejudicial legacies against the community group and cross-institutional negativity" (O'Mullane and Quinlivan, 2012) = less than minimise impacts
• In Swedish transport planning, cost-benefit analysis results (e.g. from RIA) influence investment decisions, with the most profitable or least unprofitable option usually being the chosen one. However negative non-valued environmental effects also have an effect, making it less likely that an investment will be included in a plan (Bondemark et al., 2020) = minimise impacts

5.4 Conclusion

We had assumed that the literature would come out in favour of integrated impact assessment, but instead a more nuanced picture has emerged. Morrison-Saunders et al. (2014), from an academic perspective, argue in favour of integration that different impact assessment types generate diversity at the expense of value, and create silos and confusion. However, some of the issues raised by the literature suggest that integration is not as simple as putting all of the impact assessment types in one document, or broadening out one impact assessment to encompass others. The integration process needs to address issues of terminology, culture and potential conflicts between the conclusions of different types of assessment. That said, many of the case study countries have moved to an integrated model fairly recently, and it will be important to continue to monitor and learn from the experiences of these countries as they work through the challenges of adopting a more integrated approach to impact assessment.