Scottish Welfare Fund review: final report - data analysis appendix

Annex 2

Regression analysis aims to summarise the relationship between a 'dependent' variable and one or more 'independent' explanatory variables. Logistic regression predicts the likelihood of something happening, based on the characteristics included in the model. Here, we look at the likelihood of receiving a) a Community Care Grant Award and b) a Crisis Grant award.

The tables below shows the variables that were included in each model (with separate models for likelihood of receiving a Community Care Grant and for likelihood of receiving a Crisis Grant). The variables included in a model are significant in helping to explain the odds of receiving that award, controlling for all other variables in the model. Other variables that were tested and were not significant included some benefits, vulnerability categories and types of referral. Some variables were significant in the Community Care model but not the Crisis Grants model and vice versa.

Interpreting regression models

• 'Pseudo R-Squared' – model fit: The overall extent to which the model adds to our understanding is measured using a test of the 'Pseudo R-squared'. A higher pseudo R-squared indicates which model better predicts the outcome of receiving an award). For the Community Care Grant mode, the R Square is 0.349 (Nagelkerke R Square). In linear regression, this R Square would indicate that the model explained 35% of the variation but this is not the case in logistic regression. We can conclude that the Community Care Grant model is better than the Crisis Grant model (Nagelkerke R Square 0.176) though.
• Reference categories: Logistic regression models compare different categories against a reference category. For example, in Table A-1, Aberdeen has been set as the reference category for comparing local authorities (as the first local authority alphabetically), and the other categories are a series of comparisons with this category.
• P-values – statistical significance: The column headed 'Significance', shows whether the factor is significant. A p-value of 0.05 or less indicates that there is less than a 5% chance we would have found these differences between the categories just by chance if no such differences were present, while a p-value of less than 0.01 indicates that there is a less than 1% chance. P-values of 0.05 or less are generally considered to indicate that the difference is significant. In Table A-1, for example, the figure for Dundee (vs. Aberdeen) is less than 0.05. It follows that - after controlling for the effect of all other factors in the model - the likelihood of receiving a CCG award in Dundee is significantly different from the likelihood of receiving an award in Aberdeen. The odds of applicants in Argyll and Bute (sig 0.3) and East Dunbartonshire (sig 0.8) of receiving an award were not significantly different, compared with the reference area (Aberdeen).
• Beta – direction of effect: The column headed 'Beta' indicates the direction of the effect. A positive value indicates that applicants in that category are more likely to receive an award, and vice versa. For example, Table A-1 shows that applicants in Dundee are more likely than applicants in Aberdeen to receive an award, as the Beta co-efficient is positive. In contrast, those in Falkirk are less likely to receive an award as the co-efficient is negative.
• Odds ratio – size of effect: The column headed "Exp(B)" gives the odds ratio. This indicates the size of the effect. The further above 1 that the odds ratio is, the greater the increase in likelihood of receiving an award. The further below 1, the greater the decrease in the likelihood of receiving an award. A value of 1 for the odds ratio means that a factor has no effect. Taking Table A-1 again, applicants in Dundee had a greater likelihood (or "odds") of receiving an award compared with Aberdeen – controlling for all the other variables in the model, while applicants in East Lothian had lower odds of receiving an award.

Regression model data – Likelihood of receiving a Community Care Grant award

The highest likelihood of receiving a Community Care Grant award (after controlling for other factors included in the model) was in West Dunbartonshire followed by South Lanarkshire, North Ayrshire and Edinburgh.

East Dunbartonshire, Perth and Kinross, Moray, Eilean Siar, East Ayrshire and Highland authorities were not significantly different from Aberdeen.

Receiving an award for Community Care Grants was less likely in West Lothian, Midlothian and Falkirk than Aberdeen.

Looking at application method (Table A-2), those applying online were less likely to get a Community Care Grant award than those applying by telephone. In contrast, those applying by post were more likely to get an award than those applying by telephone. There was no significant difference between telephone applications and face-to-face applications.

Those receiving help to apply or where a third party applied on their behalf were more likely to receive an award than the reference group of those who applied by themselves (sole applicant).

Applicants reporting no disability/illness and those who refused information were more likely to receive a Community Care Grant award compared with the reference group of those indicating a disability. This is controlling for all the other aspects of vulnerability in the model discussed below.

Age was significantly related to successful application, with older people more likely to receive an award.

Table A.3 shows that applications in November and December were not significantly different in terms of likely success compared with the reference month of January (with p values of 0.13 and 0.61). Likely success was higher in other months by comparison – with March applicants, for example, having the most likelihood of success followed by September, August and May.

The reference group for property type was local authority tenancy (Table A-4). Most applicants in other types of property were less likely to receive an award compared with the reference group.

Those in private rented accommodation were significantly less likely to receive an award, as were those living with friends/partners and those living with parents/family. Those in prison/detention and other institutional settings/homeless or insecure accommodation were also less likely to have a successful Community Care Grant application, compared with LA tenants. Again, this is controlling for other vulnerability factors considered below.

Applicants in a housing association (RSL) tenancy were more likely to receive an award compared with the LA tenant reference group. Other groups were not significantly different from LA tenants in terms of likelihood of success – those in the armed services (p=34) those in children's residential care (p=0.89) and staff intensive sheltered housing (p=0.54).

Applicants who were moving to an RSL tenancy or LA tenancy were more likely to receive an award, compared with those who were not moving (the reference group in Table A-5) while those moving into a PRS tenancy were less likely to receive an award. Other groups and those applying in advance of a tenancy were not significantly different from the reference group of those not moving (with p values>0.05).

The reference group for the reason for applying was 'leaving hospital' (Table A-6), with many other groups not significantly more or less likely to receive a Community Care Grant (with p values>0.05) including those leaving a care home, hostel or shelter, sheltered housing, LA care and foster care, prison or detention centre. Those less likely to receive an award included those giving 'other' reasons, those needing help to improve a home to maintain living conditions, those needing help to stay in the community and those needing to move to more suitable accommodation to prevent admission to care.

The likelihood of receiving an award is significantly less for repeat applications and where applicants have longer waiting times (Table A-7).

The model compares the outcomes for those who receive different benefits, compared with those who do not receive each benefit. Successful awards were more likely among those on Pension Credit Plus, those on PIP, Council Tax Discount, Disability Living Allowance and 'other benefits' (compared in each case to those not receiving each of these benefits). By comparison, those on Working Tax Credit (WTC) Income Based JSA, Universal Credit and Attendance Allowance were less likely to receive an award compared with those not on these benefits. Those waiting to receive benefits were more likely to receive an award compared with those not waiting.

There are a range of variables about referrals in the model (Table A-8), with those referred for debt advice and those referred for money management and the Men's Advice Line less likely to receive an award, compared with those not referred to each of these types of services. Those referred to Housing and those referred for resilience/befriending support were more likely to receive an award, as were those referred to hospital, Social Work and CAB (compared with those not referred to each of these).

Vulnerability is associated with a greater likelihood of receiving an award, with many indicators of vulnerability significant in the model (Table A-9). Those reporting no vulnerabilities were less likely to receive an award than those who reported a vulnerability. Those fleeing domestic abuse, homeless people, care leavers and kinship carers were more likely to receive an award than those without these vulnerabilities. Those with poor mental health and those with various physical health issues were more likely to receive an award, compared with those without these vulnerabilities.

Those applicants facing eviction/repossession, young people unable to live with their parents because it would put them in danger and children living with a disabled adult were less likely to receive an award that those not vulnerable for these reasons.

Regression model data – likelihood of receiving a Crisis Grant Award

The tables below shows the variables that were included in the Crisis Grant logistic regression model.

The highest likelihood of receiving a Crisis Grant award (after controlling for other factors included in the model) was in Eilean Siar, West Dunbartonshire and North Lanarkshire. Many other local authorities were also more likely to receive an award, compared with Aberdeen (the reference category), while East Ayrshire, Moray and Midlothian were not significantly more or less likely to receive an award (with p values >0.05).

Argyll and Bute, East Lothian, Falkirk and West Lothian all had a lower likelihood of success in receiving an award compared with the reference group (Aberdeen).

In the Crisis Grant model, online applications were less likely to be successful, compared with the reference group of telephone applicants while face to face and post applications were not significantly more or less likely to receive awards, compared with telephone applicants (with p=0.3 and 0.1).

A successful award was more likely where a third party applied on behalf of the applicant, while those receiving help to apply were not significantly different in success, compared with sole applicants.

Holding all the variables in the model constant, including the vulnerability indicators, those without a disability and those refusing disability information were more likely to receive a Crisis Grant award, compared with people reporting a disability. Female applicants had a lower likelihood of success compared with males, while the likelihood of success increased with age.

Looking at the reason behind the application (Table A-13) shows lower likelihood of a successful award where 'other' reasons are given, for travel, where money is lost or stolen or where they have nowhere to stay, or flooding, compared with the reference category of an 'unexpected expense'. Applications were more likely to be successful where there is a delay in the payment of benefits or a fire.

Household type was significant in the Crisis Grant model (Table A-14), though not in the Community Care Grant model. Holding all the other variables in the Crisis Grant model constant, all other household types were less likely to receive an award compared with the single person reference group, except 'other' households with no children, who were more likely to receive an award.

Applications made in March were most likely to be successful, compared with the reference month of January. In fact, there was more likelihood of in all months, compared with January, except February and April, when success rates were not significantly different.

For property type (Table A-15), those with an LA tenancy were the reference group, with most other tenures having a lower likelihood of success in applying for a Crisis Grant. The exception is for housing association (RSL) tenants who were more likely to have a successful award. The lowest likelihood of success was found among those in residential care or prison/detention centre and hospital. Those not significantly different from LA tenants, in terms of award success rates included those in the armed services, children's residential/foster accommodation, sheltered housing, supported accommodation and other insecure accommodation (e.g. B&B, caravans). Those in hostels and homeless accommodation were less likely to receive a Crisis Grant award than those in an LA tenancy, though.

Again, repeat applications had a lower likelihood of having a successful award for Crisis Grants (Table A-16). Generally speaking, people on benefits were less likely to have a successful award compared with than those not on each benefit.

Those on income-based Job Seekers Allowance and those on Council Tax Discount had a higher likelihood of success, compared with those not on these benefits, though.

Those subject to a benefit sanction had a lower likelihood of a successful Crisis Grant award, compared with those not sanctioned.

Those referred to some agencies (Table A-17) had a lower likelihood of receiving a Crisis Grant award – in the case of resilience/befriending, foodbanks, welfare rights, Social Work and hospital there was a lower likelihood of success compared to those not referred.

Applicants referred to Housing, Employability, the CAB (Citizen's Advice), Men's Advice Line and a named organisation (the Anthony O'Brien Quinn Memorial Trust) had a greater likelihood of receiving an award, compared to those not referred.

Those vulnerable for various reasons were often more likely to be successful in receiving a Crisis Grant (Table A-18). Those with no vulnerability identified were less likely to be successful than those who had some vulnerability recorded.

Vulnerabilities with a greater likelihood of receiving an award (compared with those not vulnerable for each category) included – leaving the armed forces, insecure work and domestic abuse, kinship carers and young people estranged from parents. Those vulnerable due to bereavement, mental health issues, homelessness, lone parenthood and chronic ill health also have a greater likelihood of receiving an award than people not vulnerable for this reason in each case.

Vulnerable groups less likely to receive an award (controlling for the other variables in the model) included those unable to live with parents due to being in potential danger, young people living with a disabled adult, children with young parents, people vulnerable due to family breakdown and eviction/repossession (compared in each case to those not in each group).