Sheep attacks and harassment: research
Findings from survey research on sheep worrying and wildlife attacks on sheep.
Appendix F: Cognitive Testing Findings
The cognitive testing of the survey took place between 4th and 27th March 2019. The survey questions were tested through in-depth telephone interviews with a total of twenty sheep farmers recruited to encompass a mix of regions across Scotland, a range of farm sizes (from those with 1-20 sheep up to those with 750+ sheep) and a mixture of LFA and Non-LFA land. Fifteen of the participants were recruited from the mainstage sample, while the remaining five were recruited through informal connections and networks. We split the survey into sections, asked participants the questions, then asked how they found each section. We also probed on specific points. We achieved a good mix of farm sizes, and spoke to farmers in all regions of Scotland, as outlined below.
| Farm size (number of sheep) | 1-20 | 21-150 | 151-750 | 750+ |
|---|---|---|---|---|
| 3 | 6 | 6 | 5 |
| Region | NE Scotland | Eastern Scotland | South Western | Highlands and Islands |
|---|---|---|---|---|
| 4 | 6 | 2 | 8 |
This document summarises the main findings relating to the questions. The findings - and suggested amendments and potential solutions to problems identified - should be considered in the context of the questionnaire currently being too long.
Most of the participants had experienced one incident (either a dog attack or a wildlife attack) in the past year and so were asked the detailed questions about that incident. The survey was initially taking 25-30 minutes with those participants and then 20-25 minutes when we skipped some of the questions or some of the response options. It was taking around 12-15 minutes with those who had not experienced any incidents. This is considerably longer than the average of 10 minutes which we had anticipated. It was partly because the participants were keen to talk about the incidents and the topic in general, so gave additional details and comments (which was interesting and useful - and bodes well for the response rate) and partly because they took some time to consider their responses to some questions. However, we clearly need to cut the length. We have made a number of suggestions below about what could be cut: whole questions in some cases and reducing the response options in other cases.
| Question |
Issues identified |
Proposed change |
|---|---|---|
| Q1 On what type of land are your sheep?
|
Participants found it quite difficult to choose from the answer options on this question. In particular, some commented that they found it difficult to recall the second answer option "Open hill or common grazing with all/some of lambing inbye/partly enclosed land" and asked for it to be repeated. Further, some participants commented that they thought common grazing was extremely rare, and indeed nobody described their land as "common grazing". |
We propose changing the answer options to:
We tested these shortened answer options with some of the participants and they worked much better. We also propose moving the position of this question from the beginning to the end of the survey, to follow the question "Do you have any other suggestions for measures to prevent dog and/or wildlife attacks?" as one participant said she had not experienced any attacks when what she meant was that she had not experienced any attacks on her current farm. She was thinking about her current farm because we had just asked about the type of land. Moving the question to the end avoids this issue. |
| Q3 Is there a track or path which is regularly used by dog walkers close to any of your sheep? |
Although this question generally worked well, a few respondents asked for clarification on whether they should include roads in their answer. However, it is worth noting that respondents said 'yes' even if there were very few dog walkers (e.g. two a day). One option might be to ask a follow up question about how many there are, but this would add to the length. |
We propose amending the wording of the question to "Is there a track or road which is regularly used by dog walkers close to any of your sheep?" We tested this new version of the question with some participants and it worked well. We also propose moving this question to the final section of the survey, before the above question about type of land. |
| Intro to questions on dog/wildlife attack incidents: "We'd like to ask you about the most recent incident, but we'll give you the opportunity to tell us about other incidents later on if you would like to. Please tell us about the most recent incident in [insert most recent month with an incident]." |
Some participants talked about or make reference to incidents in addition to their most recent incident in this section. |
We propose amending this instruction to emphasise that participants should only discuss their most recent incident in this part of the survey. We suggest rewording to "Please just tell us about the most recent incident" |
| Q12/Q32 Where did it happen? |
When asked to input their holding number some respondents found it difficult to recall the number in full. Amongst those who could recall their holding numbers, some gave numbers shorter than seven digits long, as they did not include the full number of 0s in their answer. |
Since we hold information on one (main) holding connected with each respondent, and most will not have more than one holding in total, it should generally not be necessary to ask respondents to provide their holding number; and the issues identified in cognitive testing can thus be avoided. We therefore propose amending this question to "Do you have sheep on more than one holding?" and only if they answer "Yes" to this question, to ask them the follow up question: "On which holding did this incident happen?" with the instruction "Please input Parish and holding number (this should be a seven digit number - include leading 0s e.g. 012/0004 if your parish number is 12 and your holding number is 4." In the online version we would provide a box with three followed by four spaces for respondents to input their holding number where necessary, to ensure respondents input the right number of digits. : _ _ _/_ _ _ _ |
| Q21/Q37 How were your sheep affected? On each row, please tell us the number affected
|
In this question, "Hefted sheep displaced" caused some confusion amongst respondents. Several asked for clarification as to what was meant by this, and one commented that it would be impossible for farmers with larger numbers of sheep to know when the sheep had moved and to link it to an attack as sheep are only counted every few weeks. Another issue with this question was that some respondents made reference to effects on their sheep from incidents other than the most recent incident when asked about their most recent incident. |
Part way through the cognitive testing, we amended "Hefted sheep displaced" to "Sheep displaced and did not return themselves", but some respondents continued to query this. Nobody cited this as an effect of a dog or wildlife attacks and we feel that where it did happen, it would be captured in the later question about time spent dealing with the incident. So we propose removing this effect from the list of answer options. To emphasise that we want to hear just about the most recent incident in this part of the survey we propose amending the question to read "Again, just thinking about this last incident with the [input species], how were your sheep affected?]? " |
| Q21/Q37 What kind of sheep was this?
|
Most participants commented that these terms did not encompass all those they would use to describe their sheep. They suggested the addition of "gimmers" and "hoggets". |
We propose updating the list to:
We tested this in some interviews and it seemed to work well. |
| Q22/Q42 Thinking specifically about this incident, approximately how long did you spend dealing with it? If you are not sure please give your best estimate |
A few participants referred to treating their sheep themselves (as opposed to solely having it treated by a Vet) in this question. |
We propose adding an answer option "Personally treating the injured sheep" |
| Q23/Q41 Thinking specifically about this incident, approximately what was the financial cost, if any of the incident to you? |
A few participants indicated that in some instances farmers may treat sheep or administer medicine themselves and this may be a cost not accounted for on the existing list of potential costs. "Value of lost sheep" does not cover a reduction in value of a sheep which has been injured (e.g. lost an eye) but not lost. One farmer said he had received £300 compensation from the dog owner. |
We propose adding "Other treatment costs (e.g. medicines/dressings) not through vet" We propose changing "Value of lost sheep" to "Lost value of sheep". We tested these amendments with some participants and they worked. Adding "If you received any money from an insurance claim and/or any other compensation, please say how much you received". |
| Q38 What species do you think was involved?
|
A couple of participants talked about attacks by "bonxies". (i.e. skua) and black backed gulls. |
We propose adding "skua" and "black backed gull". |
| Q46 Have you put any of the following measures in place to try to prevent attacks by dogs?
|
We found this list of measures took a very long time to read out over the telephone and a large number of the measures had not been taken by any participants. One measure not included however, came up spontaneously numerous times - talking to dog owners/walkers. |
To shorten the survey, and to ensure the question is relevant to respondents, we propose shortening the answer options list to:
We tested this in the latter stage of cognitive testing and it seemed to work well. The "other" answer option ensures that the results should still reflect the full range of measures taken. |
| Have you put any of the following measures in place to try to prevent attacks by wildlife?
|
A couple of the measures in this question caused confusion amongst participants. Many respondents expressed uncertainty about what "Taste aversion bait" was and none of the 20 farmers interviewed had used it. There was also considerable confusion around the "Scottish National Heritage (SNH) licensed control" with a majority of participants asking for clarification of what this was and nobody having used it. Further, only one participant had used cameras to monitor the area. |
We propose removing "Taste aversion bate", "SNH Licensed Control" and "Cameras to monitor the area" from the list of answer options. We tested this in the latter phase of the cognitive testing and the shortened list appeared to work well. As with the previous question, the "Other" answer option will give respondents the possibility to mention any measures not listed. |
| Q51 Do you think the following are increasing or decreasing…
(Increasing a lot, increasing a little, no change, decreasing a little, decreasing a lot, don't know) |
This question was very long to read out and many respondents answered "Not applicable" to most of the wildlife species cited as the species were not present in their area. |
We propose amending this question to "Do you think the following are increasing or decreasing…
We tested this new version of the question with a few participants and it worked well. |
| Q53 How effective do you think the following measures would be at reducing attacks by dogs?
(Ineffective, somewhat effective, very effective, don't know) |
In the cognitive testing, a large number of respondents wanted to give answers which differed from the "Effective/ineffective" scale, such as they variably thought the measures were unrealistic, impractical, or insufficient as the dog should in question should be put down. We also felt that the first measure was quite long to read out and a few participants asked for it to be repeated. |
We propose amending the question to: "From the list below of potential measures for reducing attacks by dogs, please indicate whether you think each should be a priority or not a priority" and amending the answer options to "Should be a priority", "Not a priority" and "Don't know". We tested these new answer options in the latter part of the cognitive testing and they worked much better. Although there may have been a risk that participants would say all of the measures should be a priority, this did not happen. We also propose slightly shortening the first measure to "Requirement to keep dogs (other than working dogs) on leads within defined areas of the countryside". |
| Q54 How effective or ineffective do you think the following measures would be at reducing attacks by wildlife or reducing the effects?
|
As with the equivalent question on dog attacks, a large number of cognitive testing participants wanted to give answers which did not fit into the "Effective/ineffective" scale. Most participants also made negative comments about the last three measures: "Stopping reintroductions/rewilding efforts", "Provision of alternative food sources for predators" and "Measures to increase the viability of lambs". These measures presented different problems. Almost all participants found it hard to answer "Stopping reintroductions/rewilding efforts" because it depended on the species (i.e. they would support some rewilding efforts and not others). Many commented that "Provision of alternative food sources for predators"" was an odd suggestion, was likely to be ineffective and potentially counterproductive. Almost all supported "Measures to increase the viability of lambs" but found it an odd suggestion because it was so obviously a good thing and something everyone would be trying to do anyway. Asking about it is therefore unlikely to provide useful data. |
We propose amending the wording of the question to read "From the list below of potential measures for reducing attacks by wildlife or reducing the effects, please indicate which you think should be priorities" and amending the answer options accordingly to "Should be a priority", "Not a priority" and "Don't know". We tested this changed wording in the latter half of the cognitive testing and it worked well. We also propose reducing the number of measures to just the first three: "Enhanced compensation schemes for losses"; "Increased powers to allow control of wildlife" and "Change the protected status of some species". |
| Are you aware of the Taking the Lead guidance published by Scottish Natural Heritage? |
This question worked, but we feel it could be taken out given the length of the questionnaire. |
We propose removing this question to reduce the overall length of the survey. |
| Do you have any other suggestions for measures to prevent dog and/or wildlife attacks |
In the cognitive testing respondents often used this question to reiterate points they had already made earlier and this question added to the length of the questionnaire |
We propose removing this question for the telephone version of the survey (we would keep it in for the online version). |
Contact
Email: socialresearch@gov.scot
There is a problem
Thanks for your feedback