Rent Pressure Zone: application requirements

Annex 2

Guidance on Methodology, Analytical Quality, Sampling and Conducting Interviews

Where local authorities gather data, or have data commissioned, they should ensure that the methodology used is also shared with Scottish Ministers as a way to help evidence that data are robust. Without such methodology, Ministers will exclude the data as evidence.

Given that an RPZ is based on rents rising by too much for existing tenants in the same properties, the focus of the data collection should be to collect information on these rent rises directly from tenants and/or landlords. Councils should ensure that data collected from tenants on rent increases is robust. Councils should be able to prove that they have this evidence if requested by Ministers.

The criteria at Annex 1 provide as much detail as possible about what needs to be included in an application. It is not possible to provide standardised thresholds for all criteria because the profile and performance of rental markets will differ widely. For example the RPZ threshold in a city region with a large, complex and fluctuating rental market is likely to be different than that in a rural setting with a smaller, more uniform, less fluctuating rental market.

However, the overarching principle across all criteria, and in the application marking scheme, is analytical quality. The choices of methodology used to gather the evidence will be crucial to achieving the required levels of quality. The dimensions of analytical quality are:

• relevance, how well the evidence informs the criteria;
• accuracy and reliability, an assessment of the accuracy of the evidence;
• time frame for evidence collection, how up-to-date the evidence is, what time periods it relates to.
• accessibility and clarity, including transparency about, and ease of access to, the data used and any supporting methodology;
• how data confidentiality, has been assessed and managed for each set of evidence used.

Using statistical sampling to determine rent increases as part of an RPZ application

The following guidance sets out some key aspects of sampling methodology and places them in the context of rent sampling. It suggests some of the issues local authorities may wish to consider but it is not prescriptive.

The choice of how to gather rent data for this exercise is likely to be determined by two key factors; the size of the proposed RPZ and the length of the time that is chosen to demonstrate a rent rise over time.

If the proposed RPZ is a very small area such as a street, then sampling may not be an appropriate or effective technique. Where the numbers involved are small it may best to simply gather information on rent increases from all existing PRS tenants in the street (like a Census would). Including all these properties ensures that the data are representative. For larger areas where it is not possible to include all properties then sampling is likely to be more appropriate.

Statistical sampling is the process by which a feature of interest in a population is estimated by measuring its value in a smaller but representative sample. The aim of sampling is to produce estimates that are as close as possible to the real values in the population. A good sample ensures that results are representative of a wider population whilst keeping the amount of data collected to a minimum.

Estimates produced from sampling are unlikely to match the true population values exactly because of the lack of information obtained for the non-sampled values. Therefore it is important to assess the likely precision of the sample estimate in relation to the population value. This precision can be calculated by the sampling error.

The aim of a successful sampling exercise is to minimise the sampling error in order to be confident that the estimate produced is good enough to be useful. If the sampling error is too great then the estimate may be too unreliable or misleading to use.

The impact of sampling errors are usually expressed in terms of confidence intervals. For example, a small sample survey may show that rents have increased on average by 5% for existing tenants over a particular period. However, if this estimate has a confidence interval of +/-20%, then there would only be sufficient statistical certainty that the true increase in rents has been in the range of 4% to 6%.

Statistical sampling in monitoring rent rises

In the context of RPZs, sampling could be one way to establish the level of rent rises in particular areas. However, care should be taken to ensure the sample is representative of the rental profile of the area and data is only collected on rent rises that existing tenants have experienced in the same properties. If a tenant has moved to a different property during the reference period, then any change in rent should not be included in the evidence.

Sampling may be helpful where there are a larger number of properties where existing tenants reside, where it would be impractical to find out about all properties. Where there are a smaller numbers other methods might be more appropriate such as asking all tenants and landlords directly about rent rises. Tenants or landlords could be asked about their current rent level and past rent levels to assess rent rises over time, for the same tenants in the same properties.

Sampling error generally depends on three things:

• Sample size. Increasing the sample size covers more rental properties in the population and reduces sampling error.
• Variability in the population values. Higher variability in the population will lead to higher variability from sample to sample, and may result in higher sampling error. For example, if an area has many different types of rental properties with a wide range of corresponding rents or if there is a large variation in rent increases applied by landlords, there will be more variability in the sample data. In such circumstances higher sample sizes should be considered to achieve the level of precision required.
• Sampling method. The way in which the members of the sample are chosen. A good sampling method will provide a sample that is representative of the population of interest, leading to lower sampling errors.

Representative samples can be achieved by using methods such as:

• Simple random sampling – in which each individual in the population has the same chance of being selected. In the case of rent rises this could mean sampling households at random from all the PRS properties with existing tenants in a given area, and finding out about the rent rises for each of the households selected.
• Stratified random sampling – in which random sampling is undertaken within clearly defined groups of the population. This can be a suitable technique to use where we need to ensure that different groups are adequately represented in the sample. In the case of rent rises this could mean sampling by the property characteristics of existing tenants e.g. house type, size, age, location etc. to ensure the sample obtained contains a representative number of properties with these characteristics.
• Quota sampling – in which units are selected, not necessarily at random, within pre-determined quotas or limits for different groups of the population. In the case of rent rises this could mean building up a sample of information on existing tenants in a particular area until a certain number of records are obtained for each property characteristic. This approach will rely more heavily on the local knowledge of rental properties and judgement of authorities to ensure a representative sample is selected.

Prior to drawing a sample, a sample frame/population needs to be identified from which the sample can be designed and drawn. This might be, for example, a Census, a survey or a Landlord Registration Database.

If sampling is used, by whatever method, it is important to assess the likely sampling error. If the sampling error is too great then the estimate may be too unreliable or misleading to use, or may not provide enough precision to be able to say for definite whether rents over time have been increased by too much.

Depending on the sample population/frame used it may be possible to identify specific addresses to contact, in which case a stratified random sample could be used and confidence intervals calculated.

Whilst it may not be possible to formally estimate sample errors for quota sampling, it should be possible to calculate sampling errors based on a corresponding stratified random sampling approach. This would provide a proxy for absolute minimum sampling errors, which in reality will be greater under the quota sampling approach.

It would also be important to assess and report on any potential sources of bias. This could include an assessment of how the sample was selected, how clustered or spread out any addresses are within the sample, and to consider any other clear sources of potential bias, for example if data items were from a single data source that only covered certain types of lets. An examination of the property profiles may also show potential sources of biases.

Local authorities may find it helpful to seek statistical expertise with regard to any sampling.

Conducting interviews as part of an RPZ application

Qualitative research could be used to produce rich information to show how rents are rising by too much are affecting existing tenants and could be used to provide evidence for the second criterion; which is that 'the rent rises within the proposed zone are causing undue hardship to tenants'.

The most appropriate method for this is likely to be interviews with existing tenants in the relevant area. In order to have this evidence considered as part of the application, a local authority will need to keep an accurate record of how the research was conducted. The following steps provide a short guide.

Preparation

Before conducting interviews:

1. Identify who the local authority will invite to participate. For qualitative research, there are no fixed requirements for creating a sample – the local authority needs to consider the range of individuals ( e.g. older people, lone parents and disabled people) and contexts ( e.g. employment status, who lives in the household) that would be included to enable the local authority to produce the evidence it needs.

2. Devise a set of questions the local authority would ask each interviewee. This ensures consistency and helps the local authority to analyse. The follow-up questions and prompts the local authority use would depend on the answers the interviewee gives, and the local authority would need to approach each interview flexibly.

3. Establish how the local authority would record each interview and how it would securely store the interview data and any personal individual data.

4. Address any ethical issues in the research. People may find the topic difficult to discuss and the local authority should aim to avoid participants becoming distressed. Individuals are never obliged to participate in research and can discontinue an interview at any time, if they wish to do so. The local authority can encourage and enable, but not force, participation. The local authority should also carefully consider what level of confidentiality it is able to offer participants. The local authority will also need to ensure safety of both the researcher and the participant.

5. Receive written informed consent from each participant. Ensure that each participant fully understands what the research is for and how it will be used. Clearly explain the level of confidentiality that the local authority can offer.

6. Establish how the local authority will analyse, use and report the findings from the interviews. The local authority must ensure they comply with legal requirements for dealing with personal and sensitive information and ensuring that research participants are not identifiable in any reporting.

7. Consider what support can be offered. It may be appropriate to signpost interviewees to where they can access suitable support for the issues raised in the interview – the local authority should be prepared to provide this at the end of the interview.

8. Consider how to enable the tenant to participate fully in the interview: what would work best for the interviewee in terms of location, timing, format and recording of the interview? This could include consideration of whether the interview should be conducted in person or over the telephone or via Skype (or equivalent).

Key questions and follow-up questions a local authority might ask

The local authority needs to establish whether rents that are rising by too much have made a material difference to existing tenants. The local authority could consider using the following question derived from the Scottish Household Survey ( SHS). The question would need to ascertain how existing tenants were coping before and after the rent increase.

Remember - people may not be coping financially for a number of reasons, but the only evidence that will be accepted is that which relates to rents that are rising by too much. The local authority could gather evidence on rent increases over time as part of the research - this can be done retrospectively, by asking tenants their current rent and their rent at a previous date/s.

The SHS-based question is as follows:

Thinking about everything together, which of these phrases best describes how you and your family are managing financially these days following the rent rise?

• Manage very well
• Manage quite well
• Get by alright
• Don’t manage very well
• Have some financial difficulties
  
• Are in deep financial trouble
  
• Refused
  
• Don’t know

If the answers to these questions (or questions like them) suggest that the family is coping or getting by financially then the interview doesn't need to proceed. However, if, following the rent increase an interviewee reports that they are:

• not managing very well, or having financial difficulties and
• this is a worse situation than before the rent increase

Then follow-up questions will be needed to explore the extent and the impact of the issue.

The areas a local authority might want to explore are:

• How they are coping financially?

Prompts could include (these are examples only and a local authority is likely to add others relevant to the local situation):

• Any changes to their income during this period
• Ability to pay rent, utility (or other) bills, purchase food
• Whether they are behind with any payments
• Have they had to cut back on anything e.g. food or heating
• Have they had to borrow money
• Are they considering moving house
• Have they used a food bank
• Have they had to stop doing anything because they can no longer afford it due to the rent increase? What? How does this affect them?
• Has the rent increase affected their health (both mental health and physical health)? How?
• Have they discussed their difficulties with their landlord since the rent increase?