An Investigative Review into the process of establishing, managing and supporting Independent Reviews in Scotland

Chapter 2: The title, remit and terms of reference of a review

The title

36. The title of a review should provide an immediate and accurate indication of the subject matter under investigation.

37. Some reviews have a short title, for example, The Commission on Women Offenders.^[33] Others are more specific, and refer to their published documentation, for example, The Independent Review of Hate Crime Legislation in Scotland: Final Report.^[34]

38. The title will usually be directed by the nature of what is to be reviewed and provided by the person who has commissioned it. If a chair has already been appointed, this may have been done in consultation with them. The terms within the title should describe the key elements of what will be considered. These key or material elements should be commonly understood by all members of the review. Finally, the title should set the tone of the review.

The Mesh Review

39. The Mesh Review was entitled: The Scottish independent review of the use, safety and efficacy of transvaginal mesh implants in the treatment of stress urinary incontinence and pelvic organ prolapse in women. The first chair confirmed that the title had already been decided and given to her by Scottish Government officials. Key words in this title included “Scottish”, “independent”, “safety” and “efficacy.”

40. We asked members if they recalled whether the terms “safety” and “efficacy” were initially included and if their meanings were defined and agreed. There was a lack of consensus regarding whether safety and efficacy were part of the original title. Some believed it was, but the majority said that it came about as a result of the first meeting, although its inclusion was not contentious. One petitioner’s testimony to the Public Petitions Committee recalled this differently saying that she had to “fight” to get the word “safety” into the title of the review.^[35]

41. As to how these critical words were defined: “I don’t think those were defined as to the way in which they were to be used.” Others agreed adding, “not to my recollection.” Some members had a clear definition of what safety and efficacy meant to them; whereas for others the definitions were less clear: “safety I think is slightly easier, but efficacy I think is a very difficult one.” One member believed that the clinicians and petitioners members were “focusing on different things” when they were using these terms. Another acknowledged that “safety” may not always be the same as “benefit” noting that the “final arbiters of benefit are between a patient, the clinician and the informed consent process.”

42. The majority of the Review members did not recall the definitions of safety and efficacy being discussed. There was some suggestion that a common understanding was reached, however this was not supported by the divergence of response from members, or the supporting evidence provided from the Group meetings.

43. A lack of definition and agreement as to how these terms should be understood caused many challenges as the review progressed. As one member observed:

“When you don’t have a common outcome measure in terms of the research and publications, you then end up with this question which comes up halfway during the Review, what does efficacy mean?”

We recommend that, where possible, a chair is involved in the decision of what the title of the review should be.

We recommend that material or key terms contained in a title should be explicitly defined and agreed by members.

The remit

44. The remit involves the areas of activity that the review will undertake to meet the terms of reference. It should sit consistently with, and expand upon, the key words contained in the title.^[36]

45. Given that the chair has the responsibility to lead and ensure that a review fulfils its remit, where possible, he or she should be involved in drafting it. We recognize that this may depend on the nature of the investigation. Whatever approach is adopted, the rationale for the remit should be clear. A process should be in place so that the chair and the members of the review are aware of the remit and agree to work towards its fulfilment.

The Mesh Review

46. The remit of the Mesh Review was:

To evaluate both the efficacy and extent of the causes of adverse incidents and complications associated with transvaginal mesh surgery for Stress Urinary Incontinence (SUI) and Pelvic Organ Prolapse (POP).^[37]

47. How the remit for the Mesh Review was decided or agreed upon remains unclear. “I don’t know. I wasn’t involved so whoever drew it up, I’ve no idea.” “We were very much given a remit.” “I don’t remember the remit being set by us.”

48. There were different recollections as to where the remit originated, with some suggestions that a “rough guide” was provided by Scottish Government officials; no one was clear. No one described making any attempt to change or amend the remit.

Was the remit achievable?

49. The majority of Review members believed it to be achievable but only if it was understood against what may be construed as narrower, clinical definitions. If this was opened up to include and represent the broader interests of the whole membership of the Review, then achieving the remit appeared more doubtful.

50. Probably “depended on what you wanted to achieve.”

“the remit was: ‘is there a place for these clinical procedures?’”

“If the remit was – ‘should we ban the tapes?’ – that was slightly different.”

51. Some, including the second chair, regarded the Mesh Review primarily as a clinical review. If that were so, she should have had influence over the direction and conclusions that would flow from that interpretation.

52. Given the wider representation of interests on the Review Group, it would be unsurprising if others expected it to consider more than just clinical guidance. Consideration should also have been given not just to the membership of the Review but also who were the intended audience? There were diverse and competing interests represented by the membership of the Mesh Review. Although this would not be unusual in the composition of any review, there seems to have been no opportunity to agree on how membership and public interests would be represented within the remit. If the remit was simply to be viewed from a clinical perspective, then perhaps the approach could have been agreed. However, given there were 26 members, many of whose interests and expertise were not clinical, this needed to be acknowledged.

53. Insufficient consideration was given to discussing and agreeing the remit. Given this lack of resolution toward a common understanding and approach, it is understandable that the remit was never going to achieve its stated aims.

We recommend that, if possible, the chair should be the principal author in the drafting of the remit.

We recommend that the interests and expertise of all members are considered when drafting and agreeing the remit.

We recommend that the rationale for the remit is clearly agreed.

The Terms of Reference

54. The terms of reference defines the purpose and structure of a review.^[38] It also provides the chair with the authority to carry out its remit.^[39] Getting the scope and content of the terms of reference right is critical to a successful outcome. Conversely, vague and confused terms of reference will result in vague and confused investigations, considerations and conclusions.

55. Terms of reference can set the ground rules for process, governance, timescales, budget and outputs.^[40] The challenge for any terms of reference is to provide sufficient focus but retain a flexibility to meet the demands of the remit. It must steer a clear, consistent and accountable path between competing or predominating interests.

56. Drafting terms of reference may be approached in different ways. Again, this will depend on the nature of the investigation. A blank sheet of paper may be welcomed by some chairs but, for most, this will be daunting. A flexible approach may allow for the commissioning party – for example, the Minister – to provide a scoping document which outlines the central issues and how they could be addressed. This can then be considered and, if necessary, amended by the chair.

57. If appropriate, it may be valuable for the chair and members to consult with stakeholders both within the review group and external to the review.^[41] This will allow them to share any concerns and outline what they expect in terms of outcomes from the review. Having such involvement from the outset may allow expectations to be expressed, understood and interests aligned. This would need to be factored in to the proposed length of a review but the fact that such a consultation has occurred from the outset may also help build trust and confidence in the process from the outset.^[42]

The Mesh Review

58. A Transvaginal Meshes Working Group (TMWG) – often referred to as the Short Life Working Group by its members – had been in existence since 2013. The Mesh Review Final Report notes that this was established at the request of Alex Neil MSP, as Cabinet Secretary.^[43] An Expert Group evolved as development of the TMWG and held its first meeting in February 2014.

59. The TMWG was initiated “to develop a clearer understanding of the issues affecting women who had suffered complications from mesh surgery.”^[44]

60. The Expert Group was established to “look at ways of improving clinical practice, including developing pathways of care for women experiencing complications and to improve the consent process to ensure women are better informed of the risks and benefits of all procedures available to treat these conditions.”^[45]

61. When she received the draft Terms of Reference for the Mesh Review, the first chair believed that it had been written originally for the Short Life Working Group. A similar view was shared by another member who speculated that the draft terms of reference was “copied over” from the Expert Working Group. The first chair did express some concerns about the extent to which one set of terms could be applicable to a different working group with a different remit. She contributed what she described as “minor changes” which she believed made the Terms of Reference more applicable to the Mesh Review.

62. Incorporating an agreed definition of key terms was discussed at the first meeting of the Mesh Review. The Minutes from the meeting reflect that.^[46]

“The Group discussed use of the word ‘safe’, agreeing that the product, location of the product, selection of patient and the procedure itself are relevant factors, supporting the view that a definition of ‘safety’ must be included within the terms of reference.”^[47]

63. No agreed definition of safety was ever subsequently included in the terms of reference.

64. Given the Mesh Review had a second chair, consideration should also have been given as to whether the terms of reference could be changed at a later stage in the Review and who had the mandate to do this. The second chair was not given an option to revise the terms of reference. In hindsight she believed that she should have been more proactive in proposing some changes in an attempt to make the terms of reference more applicable to the stage that the Review had reached when she assumed the role.

65. A few members were brought into the Mesh Review Group after the remit and terms of reference had been established, so a proportion of the membership didn’t have the opportunity to comment on the framework they were using.

66. A draft term of reference was circulated to the members of the Mesh Review but the extent to which this was discussed or amended varies in the members’ recollection. It is probable that the draft version for the Mesh Review evolved from terms of reference that had been written for the Short Life Working Group/Expert Group. Given that some individuals were members of either one or both groups in addition to the Mesh Review, it seems likely that these terms of reference may have been conflated in their recollections or otherwise gave rise to confusion. Some members joined the group at a later stage and had no opportunity to contribute to the drafting process.

67. Neither of the chairs received guidance on what their role was in relation to the drafting or amending the terms of reference. This would have been particularly confusing for the first chair who was provided with a document which had already been written, albeit for another working group. Both chairs felt, in hindsight, that they could have been more proactive in refining the terms of reference to reflect their perception of the work of the Mesh Review and their understanding of how such a review should proceed.

68. We recognise that it requires skill to be able to draft an appropriate term of reference so that focus is provided but which still allows sufficient flexibility for the investigation. That should be the proposed aim.

69. Clear and concise terms of reference are critical to the successful outcome of an independent review. Effective terms of reference provide focus and set boundaries.^[48] It scopes out the work of the review and gives a mandate to both the chair and the members of the review to fulfil its remit. A wide range of organisations have, in the past, provided suitable guides regarding the factors which should be considered in drafting terms of reference. Some guides also include a template to aid the creation of terms of reference.^[49]

70. Consideration should be given as to who is going to set the terms of reference. A chair may have been appointed who has not undertaken this type of work previously and requires support and guidance so he or she can contribute to the drafting process.

71. There should be a provision in place to address the situation where, if a material change of circumstances arises, such as the resignation of a chair, the terms of reference can be revisited and amended accordingly.

We recommend that consideration should be given as to who sets the terms of reference. For example, this could be the chair or the commissioning Minister or a combination of both.

We recommend that all members of a review should have the opportunity to contribute to the development of the terms of reference.

We recommend that the Government consider providing a guide and template to drafting terms of reference. It should be generic in nature to meet the diversity of investigations.

We recommend that a period be set aside to consult on the terms of reference. This would enhance legitimacy, promote transparency, confidence and trust in the review process. We recognize that this must be offset against other possible limitations, for example, constraints on time.^[50]