Evaluation of the local authority housing hubs approach

2 Methods

2.1 This chapter provides details of the methodology used in the evaluation.

2.2 The evaluation adopted a mixed method approach comprising:

• telephone scoping interviews with the representative of the 'lead' local authority for each Hub ⁴
• qualitative research (a combination of focus groups and telephone in-depth interviews) with local authority Hub members
• monitoring of the level and type of contact Hubs had through the Communities of Practice website
• telephone in-depth interviews with a sample of local authority Heads of Service
• secondary data analysis of national HL1, HL2 and Audit Scotland Statutory Performance Indicator (SPI) data
• data scoping with each local authority to gather information on the types of activity they are doing and the data that they are recording in relation to housing options.

Scoping interviews with Hub leads

2.3 As the first stage of the qualitative research, telephone scoping interviews were undertaken with the main representative of each of the five 'lead' local authorities. Each interview lasted around 45 minutes. The purpose of these interviews was to gather information on the working practices of each Hub, explore progress and gain the views of the 'lead' representative on ways in which the Hub had worked to date. As with all qualitative elements of the evaluation, the interviews were conducted using a topic guide designed by Ipsos MORI with input from the Scottish Government ( Annex A). All interviews were conducted between 20th April and 3rd May 2011.

Focus groups and depth interviews with local authority representatives

2.4 The second stage of the qualitative research involved a member of the research team attending a prearranged meeting of each Hub. The researcher observed the meeting and, immediately following the meeting, conducted a focus group with Hub members. The observation allowed the research team to see firsthand what happened at a Hub meeting and the way in which Hub members interacted. Hub members who were absent from the meeting were interviewed via telephone at a later date. Table 2.1 provides details of the number of participants consulted by both methods in each Hub.

Table 2.1: Number of participants in each consulted by focus group and telephone interview

Hub	Focus group	Telephone interviews
West	5	4
Ayrshire and South	4	1
Edinburgh, Lothian and Borders	5	1
North and Islands	6	1
Tayside, Fife and Central	5	1
All	25	8

2.5 The focus group discussions lasted around 75 minutes and the telephone interviews lasted around 60 minutes. Focus group and telephone interview discussions covered local authority representatives' views on how they felt their Hub had worked so far, how it was helping their local authority to implement housing options, and how Hub working could be improved. Topic guides are included at Annex B and Annex C. Fieldwork was undertaken between 10th October 2011 and 13th March 2012.

Depth interviews with local authority Heads of Service

2.6 As the final stage of the qualitative research, telephone interviews were conducted with a sample of eight local authority Heads of Service (or equivalent). The sample included the four 'lead' local authorities ⁵ and four other local authorities ⁶ selected with the aim of getting perspectives from authorities that were at different stages and facing different challenges in developing their housing options approach. These interviews had a particular focus on any cultural changes in their local authority since the inception of the Hubs and included consideration of the wider applicability of the model. The topic guide is included at Annex D. Each interview lasted around 30 minutes and fieldwork was conducted between 7th December 2011 and 31st January 2012.

Monitoring of the Communities of Practice website

2.7 To establish the extent to which the Communities of Practice website was used to share information, the evaluation also included analysis of website administration data, provided by the Scottish Government. This included analysis of:

• the number of times the website had been visited
• the profile of website users (in terms of organisation they worked for)
• the number and types of posts that were made to the website.

2.8 These quantitative measures were supplemented by subjective evidence about website usage collected during qualitative research with Hub members.

Secondary data analysis

2.9 The secondary data analysis phase of the work was designed to provide context to the study and is included in Section 1. Initial analysis also included Audit Scotland Statutory Performance Indicators (SPI) data (2009-10) and Scottish Housing Regulator Annual Performance Statistics (also 2009-10) as well as data provided by local authorities to the Scottish Housing Best Value Network as part of their Homelessness Benchmarking. That analysis, while interesting, had limitations due to the different time period of datasets and the scope of the existing secondary data. New data were released on 14th February 2012 and has been incorporated into the analysis.

Data scoping

2.10 After participating in the focus group or telephone interview covering the Hub approach more generally, local authority representatives were invited to participate in a telephone interview that focused on their approach to data collection and monitoring ⁷ . The interviews explored where data is held, what data is collected and how easy the data is to analyse and report on. Respondents were asked to reflect on their current experience and future plans, and to identify any challenges. They were also asked to reflect on the role of the Hub so far in monitoring. Findings from the data scoping stage are outlined in Chapter 3. The topic guide is included at Annex E.

2.11 Interviews were conducted with 31 members of staff involved in the five Hubs (Table 2.2). Most of the participants had attended at least one Hub meeting, although a few had a data or performance management role that meant they did not attend meetings but had an interest in monitoring. In a few local authorities, where different people were involved in the Hubs or where roles were split between operational staff and strategy or performance staff, for example, more than one interview was undertaken.

Table 2.2: data scoping interviews

Hub	Number of interviews
West	8
Ayrshire and South	6
Edinburgh, Lothian and Borders	6
North and Islands	6
Tayside, Fife and Central	5
All	31

Analysis of qualitative data

2.12 With the permission of participants, focus groups and interviews were recorded and transcribed for analysis. The analysis of the qualitative data was carried out in two stages. The first step involved becoming familiar with the data by reading through transcripts. At this stage, initial impressions or emerging themes were noted. The second stage involved identifying the key themes in the data and organising the data into categories and sub categories. Both stages were framed by the original aims of the evaluation to ensure that the analysis was focused on answering the key research questions.

2.13 This analysis and the identification of emerging key themes was informed by several brainstorming sessions conducted by the research team, and related discussions with the Scottish Government, throughout the study.

Limitations of the evaluation and interpretation of findings

2.14 As discussed in section 1.9, local authorities have been making progress towards meeting the 2012 homelessness target, with the most recent statistics showing a 20% fall in the total number of homelessness presentations from Q3 of 2010 to Q3 of 2011. However, it should be borne in mind that this evaluation is limited in the extent to which it can attribute this progress to the introduction of the Hubs; the evaluation was primarily a process rather than an impact evaluation and, at the time of fieldwork, Hubs were still in the early stages of development. Furthermore, as discussed in section 1.27, approaches to monitoring the effects of prevention activity are still being developed.

2.15 The findings of the evaluation are promising in the sense that Hub members are positive about the potential impact of the Hubs and housing options more generally. However, at this stage, it is only possible to discuss the findings around the benefits of the Hubs in terms of the effect Hubs have had on encouraging local authorities to implement housing options approaches and the extent to which these approaches are improved as a result of the Hubs. It should also be borne in mind that the findings are based on the perceptions of local authority representatives involved in the Hub process as opposed to being based on independent, objective measures of change. The views of service users are also not part of this evaluation.

Structure of the report

2.16 The report begins by providing factual information on the composition and workings of the Hubs. This is followed by a discussion of the perceived benefits of the Hubs. The report then considers the key success factors required to facilitate effective Hub working and the barriers that exist to prevent these benefits from being achieved. The report then discusses the future of the Hub approach. The report concludes with a consideration of the wider applicability of the approach to other service areas and outlines best practice to ensure anyone looking to use the Hub model could get the most from their Hub.