Gender export gap in Scotland: research

Appendix 6: Methodological note

We used a conditional difference-in-difference (CDiD) approach which combines propensity score matching (PSM) and difference-in-difference (DiD) approaches to identify the direct impacts of entering an export market. The CDiD approach is a two-step process that allowed us to estimate the difference between turnover and number of employees before and after SME entry into an export market. This approach is frequently used for policy evaluation and recommended in HM Treasury’s Magenta Book^[8].

However, there are several limitations which need to be accounted for:

1) There is a conditional independence assumption which assumes that the observable differences between the treated (exporters) and non-treated (non-exporters) group can be controlled and therefore the outcome that would occur in the absence of treatment would be the same in both cases.

2) The approach requires a rich dataset that contains the key variables that affect both the decision to enter export markets, appropriate outcome variables, and appropriate control variables. The LSBS contains a rich number of variables to be able to conduct this analysis, however, when looking at Scotland, the number of observations in a sub-sample becomes quite low.

3) The matching procedure cannot account for unobservable characteristics that can interact with the measured outcomes (e.g., managerial characteristics). Even though the LSBS contains a vast number of variables we are limited by the sample size in what we can apply.

4) When sample sizes for either control or treatment groups are small, as is the case for women-led exporters in Scotland, estimations run the risk of yielding imprecise estimates.

Despite these limitations the results are still valid as we took several steps to ensure the robustness of our analysis. Our starting point for the conditional difference-in-difference analysis was all SMEs in the UK. From here we selected firms that exported in 2018 and discarded firms that exported before 2018. We also discarded any company that says they do not know if they exported or did not export. Our treatment groups are therefore SMEs that exported in 2018 while the control group were the non-exporters.

Both groups were then split into two: observations for the whole UK, and observations for Scotland only. The remaining number of single treated firms in the UK is 3,333 and in Scotland is 182. While the number of potential control UK firms in the 2017-2020 period ranges from 5143 up to 11,579, the number of potential control Scottish firms in the 2017-2020 period ranges from 547 up to 903.

To aid further analysis and fulfil the objective of this study, these groups were further divided into two: women/equally led SMEs and male-led SMEs. The single treated women/equally led SMEs in the UK is 1,579 and in Scotland is 52. While the single treated male-led SMEs in the UK is 3128 and in Scotland is 100. For the potential control women/equally led SMEs in the UK, the number in the 2017-2020 period ranges from 1,809 up to 4,842, and in Scotland ranges from 206 to 343. For the male-led SMEs in the UK, the number of the potential control ranges between 3084 to 6288, and in Scotland, it ranges between 368 and 487.

Our outcome variable was the average turnover and number of employees. The main identification challenge for estimation purpose is to know what would have happened with the firms exporting had they not started exporting. It is important we find a control group of firms similar to those who are exporting, because those who are exporting might differ systematically from the firm who are not exporting. The key identifying assumption behind this procedure is that the treated group’s outcome variables would behave like the control group’s outcomes if there were no exportation.

For the impact analysis, we employ the difference-in-differences (DiD) matching estimator. This approach enables us to compare the changes in firm performance between exporting firms (treatment group, T) and their matched non-exporting counterparts (control group, NT). Let Y_Tj,t represent the logarithm of the outcome measure 𝑌 for treated firm _j in year t. The difference in the log of the outcome measure between year t and 𝑡−1 for treated firm j is defined as ΔY_Tj,t = Y_Tj,t – Y_{Tj,t − 1}. The same quantities are computed for the firms in the control group (NT). We then obtain the average treatment effect on the treated (ATET) by comparing these differences. The ATET is calculated as follows:

Where 𝑁_𝑇 is the number of treated firms, and 𝑌_𝑇𝑗,t and Δ𝑌_{𝑁𝑇𝑖,𝑡} are the log differences for treated and matched control firms, respectively. In the matching procedure, all variables related to the treatment assignment and the outcome were included. Each treated firm is matched with a control firm considering performance measures and observable characteristics in the before-treatment period. As matching using the exact values of all covariates is not possible, treated firms and control firms are matched using the propensity score estimated via a logistic regression model with a set of pre-treatment attributes. This method estimates the conditional probability that a firm exports based on firm characteristics.

We selected three sets of independent variables for our matching approach: location type as urban or rural, industry sector^[9], and geographical context as England, Scotland, Wales and Northern Ireland. Our goal is to identify the most similar control firm for each treated firm based on these variables to accurately estimate the propensity score. Given this objective, we are not concerned with multicollinearity among the variables, as we do not interpret the probit regression coefficients or their standard errors. Instead, we focus on the propensity score, i.e., the overall fit of the logistic regression, and more importantly, on achieving covariate balance after matching.

With the estimated propensity score, we conduct Kernel matching, matching each treated unit with multiple control units with different weights. There was 1,579 treated women/equally led SMEs in the UK. As all the firms were matched during the Kernel matching, the final number of treated firms remains 1,579 while the control firms are 9,022.

For the male-led SMEs in the UK, no treated firms were eliminated as there were matching control firms. The final number of treated firms is 3,128 while the matching approach identified 8,994 control firms.

None of the 52 treated women/equally led SMEs in Scotland were eliminated because they all had matching control firms. The final number of control firms is 610. Of the 100 treated male-led SMEs in Scotland, none were eliminated. The final number of untreated firms was 824.

After matching, we performed the DiD analysis to estimate the treatment effect. This involved three steps. Firstly, we calculated the difference in the outcome variable before and after the treatment period.

For treated firms (T):

ΔY_Tj,t = Y_Tj,t – Y_Tj,t-1

For matched control firms (NT):

ΔY_NTi,t = Y_NTi,t – Y_NTi,t-1

Then we computed the difference between the changes in outcome for treated and control firms: D_i = ΔY_Tj,t – ΔY_NTi,t. Finally, we calculated the average of these differences to estimate the ATET. The weighted difference in difference is then:

Where w_j are the weights assigned to the matched control units, NT is the number of treated units, and 𝐷_j is the DiD for treated unit j.

Results

Tables A1 – A4 present the results from the PSM and DiD for Women-led and equally led SMEs Scotland.

Tables A5 – A8 present the results from the PSM and DiD for male-led SMEs Scotland

Tables A9 – A12 present the results from the PSM and DiD for women-led and equally-led SMEs UK

Tables A13 – A16 present the results from the PSM and DID for male-led SMEs UK

Table A1: Propensity Score Matching – Kernel – Women-led and equally led SMEs Scotland

Logistic regression

Log likelihood = -177.15655

Number of obs = 662

LR chi2(2) = 10.07

Prob > chi2 = 0.0065

Pseudo R2 = 0.0276

Table A2: PSTest – Kernel – Women-led and equally led SMEs Scotland

Verification that the covariates are balanced across treated and control groups after matching.

* if variance ratio outside [0.57; 1.74]

* if B>25%, R outside [0.5; 2]

Number of obs = 317

Number of obs = 218

Number of obs = 483

Number of obs = 316

Table A5: Propensity Score Matching – Kernel – male-led SMEs Scotland

Logistic regression

Log likelihood = -315.67629

Number of obs = 924

LR chi2(2) = 2.12

Prob > chi2 = 0.3464

Pseudo R2 = 0.0033

Table A6: PSTest – Kernel – male-led SMEs Scotland

Verification that the covariates are balanced across treated and control groups after matching.

Mean t-test V(T)/

* if variance ratio outside [0.67; 1.49]

* if B>25%, R outside [0.5; 2]

Number of obs = 517

Number of obs = 379

Number of obs = 680

Number of obs = 477

Table A9: Propensity Score Matching – Kernel – women-led and equally-led SMEs UK

Logistic regression

Log likelihood = -4355.2693

Number of obs = 10,601

LR chi2(2) = 212.97

Prob > chi2 = 0.0000

Pseudo R2 = 0.0239

Table A10: PSTest – Kernel – women-led and equally-led SMEs UK

Verification that the covariates are balanced across treated and control groups after matching.

Mean t-test V(T)/

* if variance ratio outside [0.91; 1.10]

* if B>25%, R outside [0.5; 2]

Number of obs = 4,116

Number of obs = 2,404

Number of obs = 6,284

Number of obs = 3,323

Table A13: Propensity Score Matching – Kernel – male-led SMEs UK

Logistic regression

Log likelihood = -6829.5756

Number of obs = 12,120

LR chi2(2) = 182.98

Prob > chi2 = 0.0000

Pseudo R2 = 0.0132

Table A14: PSTest – Kernel – male-led SMEs UK

Verification that the covariates are balanced across treated and control groups after matching.

Mean t-test V(T)/

* if variance ratio outside [0.93; 1.07]

*if B>25%, R outside [0.5; 2]

Number of obs = 5,130

Number of obs = 2,905

Number of obs = 7,067

Number of obs = 3,673