Education and Training Research Findings No.36/2008: Evaluation of the Professional Development Programme for Educational Psychologists in Scotland

Introduction

In 2007-08, the Scottish Government Schools Directorate commissioned an evaluation of the Professional Development Programme ( PDP) for the Educational Psychologists in Scotland. The aim of the research was to carry out an assessment of the effectiveness and value for money of the PDP and options for improvement within the current level of funding. Quantitative and qualitative data was collected from Educational Psychologists ( EPs), existing PDP Steering Group members, Heads of Education Service (for the perspectives of both the Association of Directors of Education Scotland [ ADES] and the Convention of Scottish Local Authorities [ COSLA]), members of Scottish Division of Educational Psychology ( SDEP), Her Majesty's Inspectorate of Education ( HMIE), and the Scottish Government.

Main Findings

• All research participants were aware of the PDP.
• Nearly half of the EPs who took part in this study had played a part in the PDP process, and 91% of these reported that the experience was useful.
• Sixty percent had been involved in the selection of themes. All EP grades had been equally involved in the selection and the process was seen to be consultative.
• Seventy one percent of EPs had read the PDP reports, mainly as hard copies.
• The EPs most often shared and used the reports with schools, other professionals, and the Education Service.
• PDP was evaluated positively by the EPs and other stakeholders, and was considered to provide good value for money.
• The 5 main aspects of the PDP that were found to be useful to EPs and Educational Psychology Services ( EPSs) were that it led to the development of a knowledge base, collaboration with EPs in other services, development of practice, research opportunities and better service delivery for children and young people.
• Those who said the PDP was not useful to EPs and EPSs cited the following reasons: insufficient time allocation, travel barriers, topic/research themes not matching interests, variable quality of research and output, lack of focus on service delivery, variable quality of coordination and financial constraints.
• Suggestions were made for the improvement of the PDP for EPs and EPSs: improved process for selection of topics/themes, the use of more effective research methodologies, better quality of coordination, increased funding/resources, better protected time allocation, improved ways of dealing with travel barriers, broader dissemination and easier access and more effective outcomes for children and families.

Background

The Professional Development Programme ( PDP) arose from the Professional Development Initiative ( PDI) which was established in 1985 and ran until 1995. The PDP is one way of encouraging EPs to engage in research and is managed by the Association of Scottish Principal Educational Psychologists ( ASPEP) through Co-ordinators, including a Senior Coordinator, and a Moderator. A Steering Group oversees the PDP and comprises the Coordinators, the Moderator, a representative from each of ASPEP, SDEP and ADES, as well as representatives from the SGSD. The PDP involves three groups of educational psychologists from a range of Scottish local authorities working collaboratively each year on a topic chosen from one of three themes. The Support for Learning Division, part of the Scottish Government Schools Directorate, currently funds the PDP and commissioned this independent evaluation .

Aims and Objectives

The aim of the research was to carry out an assessment of the effectiveness and value for money of the PDP and options for improvement within the current level of funding. The specific objectives were to:

1. Investigate and assess the value of the PDP taking into account both the quality of the CPD and research opportunities that it offers, including its fit with other CPD opportunities for educational psychologists.

2. Assess the effectiveness of the management and organisation of the PDP including how PDP opportunities are currently publicised and the range of educational psychologists participating in the scheme.

3. Assess how the output of the PDP is accessed and utilised by educational psychologists and local authorities and the perceived impact on the professional expertise and practice of educational psychologists.

4. Assess how the output of the PDP is disseminated to other interested parties in the educational community and how it could be improved.

Methodology

One hundred and seventy EPs from at least 31 LAs responded to an online questionnaire in Phase I. In Phase II, further in-depth data was collected from 19 EPs who had participated in Phase I, some of whom were members of ASPEP. In addition, the views of the following 8 stakeholders were gathered through interviews and focus groups: the existing PDP Steering Group members, the Heads of Education Service ( ADES and COSLA), representatives of SDEP, HMIE and Scottish Government (including one representative from the Schools Directorate and two strategic officers from the Post School Psychological Services).

Findings

• Sources of information about the PDP varied with the questionnaire respondents being informed about the PDP by their line managers (57%) and others in the EPS (58%).
• Nearly half (48%) of questionnaire respondents had participated in the PDP over the years, with a representation of at least 28 LAs. The majority (91%) of the PDP participants and all Steering Group members and co-ordinators reported that this experience had been useful to them as individuals as well as to EPS.
• Changes in their experience of the PDP process over time were reported by around a quarter of questionnaire respondents while 57% felt that there had been no change. Changes were viewed both positively and negatively.
• Over half the questionnaire respondents had been involved in selection of themes (60%). All EP grades had been equally involved in the selection and the process was seen to be consultative. Other stakeholders from Phase II (namely, members of ADES, SDEP and Scottish Government) also felt that their views had been taken into consideration in the selection of themes.
• Most (71%) questionnaire respondents had read the PDP reports at least once and up to 10 times, with 61% having reading them as hard copies. Most (79%) respondents had found them very easy or easy to access although some indicated that online access was not always easy.
• The PDP reports were shared with others mainly within the following contexts: in discussions around a theme with other EPs, sharing with school and/or post-school staff and parents, for staff development within the Service, to inform practice at individual level, to inform policy/practice at authority level, to increase individual's knowledge base, to help develop further collaborative work and lastly through national /international conferences and university modules. It was suggested that this sharing can be improved, especially with the parents and children.
• The PDP was reported to be useful to EPs and EPSs. The top 5 aspects of the PDP that were found to be useful to EPs and EPSs were that it led to the development of knowledge base, collaboration with EPs in other services, development of practice, opportunities to do research and better service delivery for children and young people.
• Eighteen percent of questionnaire respondents provided reasons for the PDP not being useful to EPs and EPSs, which included, in no particular order: insufficient time allocation, travel barriers, topic/research themes not matching interests, variable quality of research and output, lack of focus on service delivery and variable quality of coordination and financial constraints.
• The following suggestions were put forward to increase the usefulness of the PDP: a focus on having a better and well planned research process, removing the travel barrier, careful selection of PDP research themes in accordance with the need to relate themes to service needs, appropriate balance between national initiatives and local needs in the selection of themes, adequate time allocation, additional funding, engagement of LAs in dissemination, and other improvements in output and dissemination.
• A minority of EPs and other stakeholders from the second phase suggested replacing PDP with something different. However, others emphasised that even if the format was changed it was important to retain certain aspects of the current PDP. It was suggested that the Scottish Government were getting good value for money, given the level of funding provided.

Conclusions and Recommendations

The PDP was reported to be very useful for EPs and EPSs, and provide good value for money. The process and outputs were seen to impact positively on the professional expertise and practice of EPs, and on service provision for children and young people. Overall suggestions were made for the improvement of the PDP research process and output, which are reflected in the following recommendations:

1. Maintain the existing consultative process for initial generation of ideas and the subsequent decision making about the PDP themes.

2. More discussion needs to take place around the methods and a clear rationale should be presented for the use of certain methods.

3. There is a need for a streamlined product which is more cohesive. Editing of the output should be undertaken, perhaps as a required task for the PDP project team. However, further resource implications of this have to be considered.

4. Project management and leadership skills should be prerequisites to becoming a PDP co-ordinator. However, project management and leadership training could be offered to PDP coordinators, where required. The PDP Steering Group (including the Scottish Government) should ensure that appropriate levels of support and resources required to undertake the role effectively are provided.

5. Maintain the focus of the PDP on collaborative research across EPSs and LA boundaries. However, it is important to ensure improved communication and opportunities for participation of EPs from remote and/or small EPSs.

6.PDP material should continue to be made available in hard copy and wider dissemination should be ensured through online sources.

7. Wider dissemination of the PDP output should take place through the LAs and EPSs and by encouraging PDP participants to publish the research in peer reviewed national and international journals.

8. More CPD opportunities should be provided through the PDP, with more effective links with the EPS development plans, the CPD framework for EPs and the annual review process for EPs.

9. The EPSs and ASPEP should consider taking responsibility for discussion sessions at local level (within the EPS or LA) and national level (through ASPEP meetings and/or SDEP newsletter).

10. The PDP Steering Group should consider evaluating the effectiveness of the process for the PDP participants and the output for all EPs on an ongoing basis.

11.PDP Steering Group should liaise with other relevant parties to look for opportunities to access alternative sources of funding and resources. This would complement the current funding provided by the Scottish Government. On the basis of the available funding and resources, they should review the format of the PDP and should consider: a reduction of themes and/or expansion of the PDP cycle, the relevance and applicability of the research to practice when selecting themes, the possibility of carrying out longitudinal studies and ensuring there are tangible links between the themes from one cycle to the next.

This document (and other Research Findings and Reports) and information about social research in the Scottish Government may be viewed on the Internet at: http://www.scotland.gov.uk/socialresearch

The site carries up-to-date information about social and policy research commissioned and published on behalf of the Scottish Government. Subjects covered include transport, housing, social inclusion, rural affairs, children and young people, education, social work, community care, local government, civil justice, crime and criminal justice, regeneration, planning and women's issues. The site also allows access to information about the Scottish Household Survey.

If you have any enquires about Education Services Research Findings, please contact

Tel: 0131 244 0894
Fax: 0131 244 5581
Email: recs.admin@scotland.gsi.gov.uk
Website: www.scotland.gov.uk/socialresearch

Online Copies

The Research findings and full report accompanied is web only and can be downloaded from the publications section of The Scottish Government website: http://www.scotland.gsi.gov.uk

This document (and other Research Findings and Reports) and information about social research in the Scottish Government may be viewed on the Internet at: http://www.scotland.gov.uk/socialresearch

The site carries up-to-date information about social and policy research commissioned and published on behalf of the Scottish Government. Subjects covered include transport, housing, social inclusion, rural affairs, children and young people, education, social work, community care, local government, civil justice, crime and criminal justice, regeneration, planning and womens issues. The site also allows access to information about the Scottish Household Survey.