Scottish National Standardised Assessments User Review 2018-2019: final report

Scottish National Standardised Assessments User Review: Year 2 – Session 2018/19 Final Report

Introduction

1. The Scottish Government commissioned the Scottish National Standardised Assessments (SNSA) as part of the National Improvement Framework.  The assessments have been in place since August 2017.

2. The Scottish Government is committed to a process of continuous improvement to enhance the user experience of the assessments and the value they provide to schools and local authorities.  To that end, we have worked on an ongoing basis with practitioners and key groups as part of our 2018/19 SNSA User Review, to obtain views and feedback from children and young people, teachers, schools, parents, and local authorities which can then be fed into a planned cycle of system enhancements.  The User Review also draws on statistics from the SNSA system.

3. This final report outlines the key findings of the 2018/19 User Review.  The report takes account of the 3 significant reviews of the SNSA published in 2019 and feeds into the planned improvement activities published in August 2019. 

• Scottish National Standardised Assessments: recommendations from P1 Practitioner Forum (18 April 2019);
• The Scottish Parliament Education and Skills Committee Report on Scottish National Standardised Assessments (23 April 2019) ; and
• The Independent Review of the Scottish National Standardised Assessments at Primary 1 (the Independent Review) led by Mr David Reedy (11 June 2019).

4. All key documents can be found on the National Improvement Framework page of the Scottish Government website.

User Review: summary of key findings

5. The key findings from the 2018/19 User Review are:  

• More than 577,000 assessments were completed by children and young people in P1, P4, P7 and S3, which equates to 93.4% of the maximum possible. 
• There is growing evidence that the information generated by the assessments and reports is being used to plan effective next steps in learning. This, in turn, is giving teachers more confidence in assessing children’s progress, with a more consistent understanding of the standards expected of Curriculum for Excellence (CfE) levels in literacy and numeracy. 
• A new 12 band long scale and 2 national norm comparators were introduced for year 2. 
• A children’s survey and a teachers’ survey were introduced providing valuable feedback.
• 91% of P1 children indicated they were happy undertaking the assessments.
• 60% of teachers who expressed an opinion reported that their experience of SNSA was better than last year 
• Training has reached more than 11,000 participants directly over years 1&2 and training and help materials are available at all times on-line, including now on GLOW.
• The service desk has responded effectively to 9,083 enquiries over years 1&2.

Background

6. The use of standardised assessment is one part of the broad general education (BGE) assessment model, which is based around teachers’ professional judgement of children’s progress and their achievement of CfE levels. There is a strong educational rationale for using a range of assessment evidence, rather than a single source, to judge children’s progress. 

7. The SNSA provides Scottish teachers with diagnostic information on aspects of reading, writing and numeracy to support the teacher’s assessment of children’s progress and to plan next steps in learning. This information helps teachers to support individual children as early as possible and avoid attainment gaps widening as children move through school.  The SNSA also provides information at a class, school and local authority level, which can be used for improvement purposes. In doing so the SNSA also provides teachers with additional, consistent evidence to use when making their professional judgement on whether a child or young person has achieved a CfE level.

Continuous improvement

8. A number of changes and enhancements were made to the SNSA for session 2018/19 in response to the findings from the first year of operation. 

Enhancements proposed and actions taken

For P1 only	Action
Questions should be re-ordered to ensure children receive less difficult questions early in the sequence.  This will not alter the diagnostic value of the assessment.	Complete.  This was delivered and included in the SNSA 2018/19 assessment
Questions should be re-designed to reduce the need for drag and drop	Complete.  There are fewer drag and drop questions in SNSA 2018/19
The practice assessments should be enhanced to ensure they reflect the full range of questions.	Complete.  Practice assessments have been enhanced for SNSA 2018/19
Questions that have caused particular issues should be removed and replaced with a question of similar difficulty but with a more familiar context	Complete.  As part of the replenishment process, questions with a more familiar context were carefully selected
Case studies on effective classroom management should be produced and shared with practitioners	Complete.  5 case studies were published in October 2018 on the National Improvement Hub
A P1 Practitioner Improvement Forum should be established.  This forum will look at all aspects of the P1 experience	Complete.  P1 Practitioner Forum ran from December to March. Report published April 2019
For all assessments	Action
Introduction of the long scale, that will show an individual’s progress in literacy and numeracy throughout their school career at the key stages.	Complete.  Included in 2018/19 SNSA with previous reports transposed to long scale in April 2019
Reports should be enhanced to take account of the new national norms established as part of the norming and equating studies	Complete.  Included in individual reports in SNSA 2018/19
Around 1/3 of questions should be replenished.	Complete.  Revised questions agreed and signed off by ES staff for SNSA 2018/19
The public facing website should be updated to provide up-to-date information for parents and practitioners on the assessments	Complete.  SNSA website and user platform updated regularly with new feed and highlight ‘tiles’.
The SNSA training package should be refined and extended	Complete. 2 courses expanded to 5 courses for SNSA 2018/19
Feedback from children and young people on their experience should be systematically collected at the end of each assessment.	Complete.  Survey on children’s attitude to assessments included in SNSA 2018/19
An annual survey of user feedback should be conducted	Complete.  Survey for teachers available from February 2019
The accessibility guidance should be further expanded with a range of developments to enhance the experience for learners with ASN	Complete.  Accessibility guidance expanded including new webinar for teachers
Case studies should be shared with practitioners on interpreting the data provided by the SNSA and how this can be used for improvement purposes	Complete.  Case studies published as above and new training course on using data for improvement
An assessment summit should take place in session 2018/19	Ongoing.  Assessment summit planned for autumn 2019

2018/19 Feedback from the SNSA system

Completed assessments 

9. At the end of the academic year the total number of completed assessments was 577,385.  The level, and pattern, of activity is broadly in line with that observed last year. Again, most activity was seen in May, with a single day peak of 16,600 assessments. There is however early signs of a movement to do assessments earlier in the year with increased numbers being done in November and March. As teachers become more familiar with the diagnostic purpose of the assessments the pattern of presentation may continue to change.

Figure 1: Bar chart showing distribution of assessment sittings over the academic year in 17/18 and 18/19

10. With the introduction of learner feedback questions we would expect a very marginal increase in assessment duration, all other things being equal.  Our analysis for 2018/19 suggests that:

• P1 assessment duration has remained constant (24 minutes on average)
• P4, P7 and S3 writing has shown little change (two increased marginally, one decreased)
• P4, P7 and S3 reading and numeracy durations have increased across the board by around 10-15%.  

11. This last point may be attributable to an adjustment in the difficulty of the questions after psychometric analysis of last year’s results.

12. Based on a theoretical maximum number of assessment sittings (i.e. every learner in every eligible school completing all assessments), the final completion rates were as follows: 

Figure 2: Bar chart showing percentage completion rates across P1, P4, P7 and S3 based on school roll

Training

13. The training programme for professionals in 2018-19 was expanded from last session’s two courses to five courses. As before, training sessions were negotiated with each local authority to ensure local needs were met. The programme has continued to be very popular, again reaching a significant number of practitioners and remains very highly evaluated.

• Course 1: An introduction to the SNSA (operation, presentation advice and guidance)
• Course 2: An introduction to interpreting SNSA data (looking at SNSA reports and their articulation with wider assessment evidence)
• Course 3: An introduction to the long scale (an overview of the newly introduced long scale from P1 – S3)
• Course 4: Using SNSA data for school improvement
• Course 5: Using the SNSA with children and young people who have additional support needs or are learning English as an additional language

Training figures from August 2017 – June 2019 

14. Whilst we do know the overall number of practitioners who have undertaken SNSA training sessions led by SNSA Support Officers (11780 as of 30/06/19), the Scottish Government does not hold detailed information on the individuals who have taken part.   Each local authority has responsibility for deciding which practitioners should attend SNSA training sessions – and information about these individuals, their role in school etc., is retained by the authority.  As well as training events led by Support Officers, SNSA training is also available online in a variety of formats, and tutor training materials are provided for practitioners wishing to cascade information to others within their own schools/authorities. Different authorities have adopted differing approaches to ensuring that practitioners have the necessary knowledge and understanding to undertake the assessments. These include:

• face to face (539 events, 10788 participants)   
• webinar events (71 events, 1160 participants and 1732 views of the recordings)

15. Practitioners are also able to access SNSA training materials from the SNSA platform and GLOW. SNSA training videos were viewed a total of 18014 times across both platforms during 2018/19 

SNSA training evaluations: August 2018 – June 2019

16. Training evaluation forms were completed at the end of each session for each of the courses available in session 2018-19. Course evaluations across all five course remained consistently high, with on average more than 90% of responses being satisfied or very satisfied that the training courses delivered on its intentions. In addition, a new evaluation was conducted this session looking at practitioners attitudes to the value of the SNSA before and after the training programme. 

Figure 3: Sample chart showing distribution of face-to-face course evaluation responses

17. In the analysis of impact documented in Impact of training on perceived diagnostic value of SNSA (ACER, April 2019) SCHOLAR finds that: 

• The number of practitioners indicating a positive perception of the diagnostic value of SNSA rose from just over half, before training, to almost 90% after training. This is significant and underlines the importance of access to training for teachers.

Action Point 1: Scottish Government and ACER will work with local authorities to develop the range, reach, and frequency of SNSA training programmes and a new training plan will be created for the start of the coming academic year.  This aligns with the evidence in the Independent Review, which recommends an expansion of the frequency of professional learning opportunities and training in all local authorities. 

Update: over 290 training events have been planned for session 2019/20 -, an increase on the previous session (258). 

Service Desk

18. The service desk has handled a large number and range of calls. Feedback about the service desk has been consistently positive, with users commenting on how quickly issues have been resolved as well as the pleasant and helpful manner of the staff.

19. To the end of May 2019, the service desk has received 2742 enquiries from teaching and local authority staff for this academic year. 95% of calls were resolved within 1 hour and 98% of calls were resolved within 2 hours. Less than 2% of calls took longer than 2 hours to resolve. 

20. Common themes of the calls received include account provisioning and password resets, assessment administration guidance, and supporting staff with generating and analysing report data.

Children and young people survey

21. Since November 2018, the SNSA has been collecting learner feedback at the end of each assessment sitting, and all users are asked to evaluate their experience.  This ranges from asking P1s to rate their experience by clicking on a happy or sad face, which is a very common self-evaluation technique used in classrooms with P1 children and is regarded as age appropriate by education specialists, up to more sophisticated questions for P7 and S3 about the difficulty of the questions and the ease of use of the platform itself.  In common with all other SNSA questions, there is no compulsion to answer however, since the introduction of the feature, more than 98% of learners have opted to provide feedback.  Over 514,000 responses were received and a sample of these is presented below, derived from P1, P4, P7 and S3 responses respectively.

Figure 4: Bar charts showing representative learner feedback about enjoyment, question difficulty and ease of use across P1, P4, P7 and S3

22. In line with other reports, we are also able to monitor responses based on learner characteristics such as gender, free school meal entitlement, and additional support needs. 

23. Typically we do not see a significant divergence in the experience of learners based on these characteristics, eg S3 learners with and without English as an Additional Language

24.

Figure 5: Bar chart showing S3 learners' perception of the SNSA segmented by those with and without English as an Additional Language

25. and P4 learners with and without Additional Support Needs

Figure 6: Bar chart showing P4 learners' perception of the SNSA segmented by those with and without Additional Support Needs

Action Point 2: Scottish Government working with local authorities will ensure that teachers and schools have access to the SNSA survey feedback responses from their children and young people.  This will be available to them from the start of the new school session in August 2019.

Update: all local authorities have access to the survey feedbacks for their schools

Teacher survey

26. In February 2019 a survey was issued to all current school and local authority staff who had interacted with the SNSA since its launch.  When the survey closed at the end of the academic year 1,474 responses had been received, with representation from all
32 local authorities.

27. Responses were very positive around the ease of use of the platform and the various means of support available to staff. 

Figure 7: Bar chart showing how easy staff found it to access learner log-in details

Figure 8: Bar chart showing how easy staff found it to generate reports within the SNSA

Figure 9: Bar chart showing how easy staff found it to navigate the SNSA system

Figure 10: Bar chart showing how easy staff found it to assign assessments to learners

28. Staff were also asked to respond to the statement “I was able to administer assessments to children or young people with ASN/EAL using existing day-to-day classroom tools or strategies”, with the following results:

Figure 11: Bar charts showing staff confidence in administering assessments to EAL and ASN learners using existing classroom tools or strategies

29. Where overall perception of the SNSA is concerned, the responses to the following two statements demonstrate a positive trend but are also indicative that more work needs to be done.

Figure 12: Bar chart showing staff perception of the SNSA in relation to the previous year

Figure 13: Bar chart showing staff perception of the value of SNSA in informing learning and teaching

30. While these responses show an increasingly positive trend, there remains a significant number of teachers who have not had an improving experience. There are also teachers who are not yet using the SNSA as an improvement tool.

Action Point 3: The need to improve the experience of the SNSA is also reflected in the findings of the Independent Review and, as a result, the Scottish Government and Education Scotland will continue to work with local authorities and schools to develop case studies and practical guidance on using the SNSA for improvement purposes.  This support material will be published on the National Improvement Hub and the SNSA website by March 2020.  

P1 Practitioner Forum

31. The establishment of the P1 Practitioner Forum was one of the commitments of the Scottish Government’s 2017/18 SNSA User Review.  The Forum met from December 2018 to March 2019 and was chaired by Professor Sue Ellis from the University of Strathclyde. 

32. The purpose of the Forum was to bring together practitioners with direct experience of the P1 SNSA to make a series of recommendations offering advice to teachers, schools, local authorities, the Scottish Government, and Education Scotland on the P1 SNSAs. 

33. The majority of members were primary one practitioners nominated by local authorities and professional networks. Other members were school leaders with responsibility for administering SNSAs, representatives from stakeholder groups, including the professional associations (EIS and AHDS) and independent bodies, academics and researchers. 

34. The Forum involved practitioners with a balanced range of views on the SNSA.  Each meeting received presentations from a number of different speakers with relevant expertise but with a number of different backgrounds and perspectives. 

35. There were four main areas of discussion at the meetings: the purpose of the SNSA; the SNSA fit with play-based pedagogies; the implementation of the SNSA; and using the SNSA data. 

36. The P1 Practitioner Forum Report, produced by Professor Ellis, was published in 18 April 2019 and provides an insight into each of the areas discussed at the meetings as well as making a number of recommendations for action by the Scottish Government, Education Scotland, local authorities and others.  All the actions were suggested and agreed by members of the Forum. The report also provides useful support and practical advice for teachers on P1 SNSA.  The Report was formally launched at two events for teachers, local authority staff, and independent bodies at the Scottish Universities Insight Institute on 25 April, and at the University of Aberdeen on 29 April.  

37. The Scottish Government has published its response to the Report, which addresses each of the Forum’s recommendations in turn and sets out, where appropriate, the action that will be taken. 

38. Further information on the P1 Practitioner Forum can be found on the National Improvement Hub. 

School User feedback

39. In addition to the feedback generated through the SNSA platform, the Scottish Government has continued to seek direct feedback from teachers and local authorities who administered the SNSA and received reports throughout the 2018/19 session.  In spring 2019, Scottish Government officials visited 16 schools and 5 focus groups spread across Scotland to see and hear first-hand the experience of the assessments. 

40. The school visits and focus groups were selected to represent a variety of local authorities in terms of demography and experience of using standardised assessments. They consisted of a mixture of headteachers, depute headteachers, principal teachers and class teachers. The discussions were conducted around a standard set of questions to provide a consistent framework for discussion. They were very helpful in gaining qualitative understanding of the user experience of the SNSA. 

Summary of feedback

41. Overall the feedback was encouraging, and showed good progress from session 2017/18.  While some concerns still remain on the implementation of assessments in P1, almost all reported positive progress since year one. There has also been overwhelmingly positive feedback on the P4, P7 and S3 assessments. 

42. The key themes explored mirror those of the P1 practitioner forum. These are:

• The purpose of the SNSA
• The SNSA fit with the P1 (play based) pedagogy
• The implementation of the SNSA, including impact on children and young people
• Using the SNSA data

The purpose of the SNSA

43. School leaders are generally clear about the purpose of the SNSA. There still remain some concerns among classroom teachers that the SNSA may be used as a high stakes assessment and for accountability purposes. The majority, however, indicated they understood that the primary purpose of the SNSA is diagnostic and formative. Confidence in this purpose is growing based on the way SNSA information is being used by school leaders, local authorities, Education Scotland, and the Scottish Government. 

44. Schools and teachers welcomed the focus on providing diagnostic information on children’s progress in literacy and numeracy. They found this very helpful and supported them in making good judgments about next steps in learning. Almost all felt the SNSA provided information that led to good reflective questioning on interventions and improvement strategies

45. As understanding of the purpose grows, there has been more discussion within schools on the appropriate timing of carrying out the assessments. While this is at an early stage, more schools are now considering carrying out the assessments earlier to give time for interventions. Others discussed the value of using them as part of the transition process.

“We are very clear that the purpose is diagnostic and we find the diagnostic information really helpful” (Primary Headteacher – Orkney)

46. Some school leaders were not yet confident that all classroom practitioners were clear about the purpose of the SNSA. They felt that further support and clarity for class teachers would be helpful from schools, local authorities, and the Scottish Government. For many it was a matter of confidence, trusting in the stated diagnostic purpose rather than a hidden ‘accountability’ purpose. Some felt the media coverage led to the confusion.

“Not every teacher trusts that this is not a judgment on their teaching!” (Primary Headteacher– Renfrewshire)

Action Point 4: The Scottish Government will work with Education Scotland to provide schools, local authorities, and other stakeholders with a clear and definitive statement on the purpose and use of P1 SNSA for the start of the new school session in August 2019. This aligns with recommendations made by the Independent Review and the other recently published reports.

Update: The statement on the Purpose and Use of Standardised Assessments was published in August and is available on the Scottish Government website

The SNSA fit or not with P1 (play-based) pedagogy

47. A number of schools visited used play-based learning as the preferred pedagogy in P1. This enabled good dialogue in terms of the fit of SNSA with play based approaches.

48. There is not one, single, play-based pedagogical approach in schools and therefore practice was varied.

49. Teachers and school leaders were clear that play-based learning is a pedagogical approach that better engaged young children and developed a range of important life skills. This approach did not, however, mean a reduction in standards, indeed most indicated that standards of learning improved. 

50. The SNSA was seen as a valuable way of assessing the progress of children towards the agreed standards in literacy and numeracy. Schools will use a range of assessments for children, many of them observational, however more formal assessments are also valuable particularly when they are applied across the country.

51. There was a consistent view that the SNSA can fit well with a play-based approach if managed carefully. This is also evident in the Independent Review which said that:

 “.. it seems that some .. conflate assessment and pedagogy, seeing the ‘moment of assessment’ as the same as a pedagogical approach. However, there need not be any disparity between a play-based approach and P1 SNSAs

There are strong examples of schools where headteachers and teachers operate a play-based approach and find no incompatibility between that and the P1 SNSA.”

52. Various different methods were observed to integrate SNSA with play-based approaches.  Most used a station approach whereby the SNSAs were administered as one of the ’play’ stations, with the teacher or member of staff. This worked well and was done individually, in pairs or in small groups.  Some preferred to deliver the SNSAs on a 1:1 basis where the children sat with the teacher, pupil support assistant or senior leader. 

53. Most teachers reported that this approach, integrating the SNSAs into a range of activities, worked very well. It was flexible and led to children finding the assessments straightforward and part of their normal classwork.

“The children were really relaxed and treated the SNSA as just another activity” (Class teacher, West Dunbartonshire)

“That was good fun – can we do it again?” (P1, South Lanarkshire)

54. There were others, however, who found the length of the assessments and the supervision required challenging.

“The Literacy assessment is very long, children are not used to sitting at a computer for this length of time” (Deputy Head Teacher, East Dunbartonshire)

55. Some schools asked that further work should be done to illustrate and share the range of approaches used within a play-based pedagogy.

Action Point 5: The Scottish Government in partnership with stakeholders, including schools and early years practitioners, will develop guidance and examples of the ways in which a play-based approach to learning and teaching can accommodate administration of the P1 SNSA. This will be made available by March 2020. This aligns with recommendations made by the Independent Review, and the P1 Practitioner Forum Report.

Implementation of the SNSA, including the impact on Children and Young People

56. Most schools found implementation in the second year easier than in the first. Now that schools understood the challenges of carrying out the SNSA, they had forward planned more carefully and found ways to improve the way the assessments were administered.

57. Schools and teachers were unequivocal that children do not, and should not, find the SNSA distressing. They were clear that it was the responsibility of the school and teacher to create a supportive and relaxed atmosphere for the assessments. Teachers understood that, if a child was getting anxious, then the assessments should be stopped.

“The children saw it as a game, they enjoyed doing the assessments.” (P1 Teacher – Highland)

58. This is supported by the findings of the Independent Review: 

“.there was scant evidence of children being upset by taking the P1 SNSA.  Where this occurred, it was mostly because children were taken away from their usual learning context and were working with unfamiliar staff. The context of the assessment can make a difference to how confident or worried the children might be.”

59. The case studies and other discussions between teachers, such as the P1 Practitioner Forum, had shown that a variety of methods could be used. This freedom to suit local situations was well received. The ability to choose different approaches as to when and how the assessments could be undertaken, along with the opportunity to stop and start the assessments, was welcomed and reduced the pressure on schools and teachers. 

“We appreciate the trust given to us to administer the assessments in the best way for our children” (Head Teacher - East Dunbartonshire)

60. Schools and teachers spoke very positively of the support made available through the SNSA service desk, with issues dealt with quickly and effectively. Further, those who had accessed the training or support materials found these to be very helpful.

61. Those schools committed to one to one support for children (P1s) found that the assessments took a long time to deliver and tied up staff for a considerable period of time. It was acknowledged that this was a similar experience to previous assessments undertaken in many schools.

“We support class teachers to do the assessment one to one with their children, however it takes a long time and is very difficult to cover, especially with large classes” (local authority officer – North Lanarkshire)

62. IT issues remain a challenge in some schools. More had moved to tablet and touch devices that the children found easy to use, however this was not universal and some children were still struggling with mouse control, scrolling and drag and drop questions. Improvements in the assessment design were noted, however more needed to be done particularly to support P1 digital skills.

63. The levels of difficultly, particularly with P1 literacy assessments, still concerned some teachers. The extended reading for some questions and the time taken for children to process or decode remained a concern. The type of language or font used was also a concern for a small number of teachers, as it was unfamiliar to children in their schools. Most found the P1 numeracy assessment straightforward.

“Children found it difficult to decode long or unfamiliar words, it’s just not the way they have been taught” (PT Primary, South Lanarkshire)

64. These issues align with recommendations from the Independent Review and the P1 Practitioner Forum report.

Action Point 6: The Scottish Government will work with local authorities and schools to develop case studies and practical guidance on using the SNSA. This will include how to manage the digital skill challenges of P1 SNSAs. This is linked to Action Point 3 above.

Action Point 7: The Scottish Government/Education Scotland will continue to review all content for the P1 literacy assessments with ACER, including consideration of the language (see Action Point 10), alignment with the early level benchmarks, levels of difficulty and the overall length of the assessment.

65. Although the potential for reducing the number of questions within the P1 literacy assessment was not highlighted as a particular issue in the User Review, it was raised as part of the Independent Review.  This is a complex matter however it has been agreed to reduce the length of the P1 literacy assessment from the start of the 2020/21 school session. 

Using the SNSA data

66. Schools were beginning to explore a variety of ways to use the data. Increasingly data was being used by all teachers involved in the SNSA and not just the leadership teams. 

67. Teachers welcomed the instant feedback and were keen to see straightaway how their children had performed. The detailed breakdown of the reports gave very useful insight on children’s strengths and areas for further development.

68. The long scale and the national norms were seen as helpful enhancements, giving teachers more detailed information and an understanding of how children broadly compared across the country.

69. Where the information was broadly similar to what was expected, this was seen by most as building confidence in teachers to trust their own professional judgments. This was deemed by many as very useful in helping teachers confirm their views. A small number of teachers, however, still felt that the assessments didn’t tell them anything they didn’t know already and therefore were not worth the effort to administer them.

70. The diagnostic information was beginning to be used to help plan next steps and improvement strategies. Many schools were using SNSA data as part of improvement discussions, along with all their other evidence, leading to interventions or support for children. Some schools were discussing this at a cluster level or in transition discussions between stages and schools.  

71. Most schools were using information from the SNSA as part of the reports they provided to parents. A variety of approaches were used depending on the local arrangements however almost no school reported parental concerns, as long as they had effectively communicated with parents. Some commented that the national media coverage had not helped with parental confidence.

“There is much more useful data, giving us ability to see patterns for following up. It is much more informative than previous assessments”. (Headteacher Inverclyde)

“Assessing children’s progress is a complex process, a jigsaw, and SNSA are a part of this” (Headteacher South Lanarkshire)

72. Not all teachers were using the information provided from the SNSA. In some cases the data was being analysed by the school leadership before being passed back to class teachers. Many schools were keen to get more advice and training on how to understand the data and use the data for improvement purposes. Some teachers would welcome a clearer understanding of how the information in the reports related to the benchmarks and achievement of the CfE levels.

73. These issues align with recommendations from the Independent Review and the P1 Practitioner Forum Report.

74. As described in Action Point 3 above, the Scottish Government and Education Scotland will continue to work with local authorities and schools to develop case studies and practical guidance on using the SNSA data for improvement purposes. 

Local authority engagement

75. Discussions were also held with more than 20 local authority Directors of Education or their representatives to obtain feedback on the operation and experience of the SNSA at authority level. 

76. Local authorities appear clear on the purpose of the SNSA and are confident that it should be used as part of a suite of evidence that supports teachers’ judgement in determining achievement of a CfE level. They see no contradiction with this and its diagnostic value to inform teachers at individual pupil level and across a cohort, although the use of this data is still at a very early stage.

77. The introduction of the long scale was welcomed by all local authorities as they saw it as providing more detailed information on children’s progress and had potential to track this progress over time.

Local authority stakeholder feedback

78. Very few of the local authorities had received concerns from parents or staff. Where this was the case it tended to be very small numbers of parents who requested that their child did not undertake the assessment and this was limited to P1. Where any concerns arose these were dealt with at a school level.

79. Some local professional associations had expressed concerns about the SNSA along the same lines as national associations, but this has not impacted on the delivery of the SNSA. The Local Negotiating Committee for Teachers discussed and agreed the process of administering the SNSA. 

80. Indeed almost all local authorities visited expressed a positive attitude to the SNSA and were clear it had caused few issues across their schools. Some did note, however, that there was some local political concern, particularly with the assessment of P1s.

Use of other standardised assessments

81. Local authorities viewed this as a transition period from the standardised assessments they had used previously to the SNSA. Almost all said that over the next couple of years previous assessments would be replaced entirely by the SNSA.

82. Some local authorities expressed a wish to see an alignment of the information from the SNSA with that gained from previously used standardised assessments in order effectively to track improvement over time.

Action Point 8: The Scottish Government will work with local authorities in autumn 2019 to help identify barriers to the replacement of previous standardised assessments and to develop potential solutions.    

Assessment windows

83. Local authorities were moving away from prescribed assessment windows as the purpose of the SNSA became clearer. Those who did advise on assessment periods emphasised the importance of direct consultation with schools to agree the best way forward, and this was seen as a practical and helpful way of implementing a new assessment with appropriate training and support available to classroom teachers. Some local authorities were leaving this entirely to the discretion of schools. 

Action Point 9: The Scottish Government will restate its clear policy position that it is for teachers, in consultation with schools and local authorities, to decide when SNSAs should be presented to learners.

Guidance, support and training

84. All local authorities felt very well supported by the help desk and almost all were very satisfied with the quality of training. It was generally felt that the training was particularly effective when delivered face to face, though increasing use was being made of the webinar approach. The importance of training in understanding the data and how to make best use of it was seen as particularly important. It was acknowledged there are still large numbers of teachers who will not as yet have had the opportunity to benefit from direct training and relied on the on-line support.

85. As set out in Action Point 1, the Scottish Government and ACER will work with local authorities to develop the range, reach, and frequency of training programmes and a training plan will be produced for the start of the academic year.

Education Scotland 

86. Education Scotland has overall responsibility for the quality assurance of the content of the SNSA questions.  This task is key in ensuring that the SNSA aligns with the CfE benchmarks for literacy and numeracy.

87. All SNSA assessment items are quality assured by an Education Scotland review group, comprising of senior literacy and numeracy officers, Attainment Advisors, HMI and members of the inclusion team.  Meetings are organised in conjunction with ACER and Scottish Government, while other stakeholders such as Twig and SCHOLAR have attended as observers. In the course of meetings, assessment items are reviewed in light of aspects such as the cultural appropriateness of texts and contexts; technical accuracy; perceived difficulty and suitability for the target age group; clarity of wording and layout, and the extent to which they reflect classroom practice.  As part of the annual refresh more than one third of assessment items (43%) were replaced from 2017/18 to 2018/19.

88. The content of the 2019/2020 was agreed once Education Scotland and Scottish Government were satisfied with the quality and functionality of the assessments.  To complement the review process, literacy and numeracy reference groups have been set up to consider broader aspects of the SNSA assessment construct, such as the range of item difficulty and the language of item descriptors. 

89. An increasing focus of the review process this session has been on the wording of item descriptors, as it is felt that this needs to be more representative of the language of the Scottish curriculum.  This is important if item descriptors are to serve their purpose of providing practitioners with a clear impression of the progress of a learner or cohort of learners in specific aspects of literacy and numeracy.  There is a need to continue writing assessment items specifically for the Scottish curriculum and context, for example to reflect the experience of learners from less affluent backgrounds, as well as to develop the adaptive nature of the assessments to reflect the capacities of both more and less able learners

90. Two webpages have been constructed on the National Improvement Hub to provide information and guidance on the SNSAs:  

• One page contains written case studies describing how a number of primary schools implemented the assessments in P1, and providing practical advice on their administration.  In addition, short videos, in which two headteachers talk about their experience, support this resource.  
• A second National Improvement Hub webpage is devoted to the content of the P1 Practitioner Forum.  This page includes film footage of each main presentation and provides a comprehensive impression of the agenda of each meeting.

During session 2018/19 SCHOLAR has enhanced its input at Quality Assurance and Moderation Support Officer (QAMSO) events by delivering an SNSA workshop to a large group of returning QAMSOs.  In addition, an online webinar was also delivered with a recording made available through the QAMSO online community. These processes align with recommendations from the Independent Review.

Action Point 10: Education Scotland will continue discussions with ACER around the language used in item descriptors, with a view to aligning the language of the descriptors more closely with the language of CfE.
Update: The item descriptors for the 2019/20 assessments are more reflective of the language of the Scottish curriculum

Summary of planned enhancements

As part of the continuous improvement process, and in light of the responses received from the different sources of evidence, a number of enhancements will be planned and implemented for the SNSA during the third academic year of implementation.  These actions are identified in the text above but are summarised here by theme for ease of reference.

Action – in relation to purpose

Action Point 4: The Scottish Government will work with Education Scotland to provide schools, local authorities and other stakeholders with a clear and definitive statement on the purpose and use of P1 SNSA for the start of the new school session in August 2019. 
Update: The statement on the Purpose and Use of Standardised Assessments was published in August and is available on the Scottish Government website

Action Point 8: The Scottish Government will work with local authorities in autumn 2019 to help identify barriers to the replacement of previous standardised assessments and to develop potential solutions.    

Action Point 9: The Scottish Government will restate its clear policy position that it is for teachers, in consultation with schools and local authorities, to decide when SNSAs should be presented to learners.
Update: The statement on the Purpose and Use of Standardised Assessments was published in August and is available on the Scottish Government website. This includes a restatement of the policy position above.

Action – in relation to support and training

Action Point 1: Scottish Government and ACER will work with local authorities to develop the range, reach and frequency of SNSA training programmes and a new training plan will be created for the start of the coming academic year This aligns with the evidence in the Independent Review, which recommends an expansion of the frequency of professional learning opportunities and training in all local authorities. 

Update: over 290 training events have been planned for session 2019/20, an increase on the previous session (258).

Action – in relation to content

Action Point 7: The Scottish Government will through Education Scotland continue to review all content for the P1 literacy assessments with ACER, including consideration of the language (see Action Point 10), alignment with the early level benchmarks, levels of difficulty and the overall length of the assessment. 

Update: The 2019/20 assessments were approved by Education Scotland and have been live since August 2019

Action Point 10:  Education Scotland will continue discussions with ACER around the language used in item descriptors, with a view to aligning the language of the descriptors more closely with the language of CfE.

Update: The item descriptors for the 2019/20 assessments are more reflective of the language of the Scottish curriculum

Action – in relation to implementation

Action Point 2: Scottish Government working with local authorities will ensure that teachers and schools have access to the SNSA survey feedback responses from their children and young people.  This will be available to them from the start of the new school session in August 2019.
Update: all local authorities have been given access to the survey feedbacks for their schools  

Action Point 3: The need to improve the experience of the SNSA is also reflected in the findings of the Independent Review and, as a result, the Scottish Government and Education Scotland will continue to work with local authorities and schools to develop case studies and practical guidance on using the SNSA for improvement purposes.   This support material will be published on the National Improvement Hub and the SNSA website by March 2020.  

Action Point 5: The Scottish Government in partnership with stakeholders, including schools and early years practitioners, will develop guidance and examples of the ways in which a play-based approach to learning and teaching can accommodate administration of the P1 SNSA. This will be made available by March 2020. This aligns with recommendations made by the Independent Review, and the P1 Practitioner Forum report.

Action Point 6: The Scottish Government will work with local authorities and schools to develop case studies and practical guidance on using the SNSA. This will include how to manage the digital skill challenges of P1 SNSAs. This is linked to Action Point 3 above.