Chinthaka Balasooriya, Scientia Education Academy Fellow

Title: The Formative Peer Review of Teaching Project

(updated version May 2021)

Grant team: Chinthaka Balasooriya, Patrick Rawstorne, Reema Harrison, Lois Meyer, Husna Razee 

Advisory team: Gary Velan, Peter Harris, Nalini Pather, Richard Vickery, Tony O’Sullivan 

Introduction 

The Formative Peer Review of Teaching project was funded through a Scientia Education Investment Fund Grant.  This project led to the design, trial and implementation of a Formative Peer Review of Teaching (FPRT) process at UNSW. Following successful refinement and implementation at UNSW Medicine, the process generated wide interest across the university. The project team received invitations from four other faculties to expand the process to their contexts.  

Theoretical background: The literature highlights the importance of implementing FPRT processes in concert with a summative peer review system (Gosling 2014; Yiend, Weller & Kinchin 2012).  A review of the literature also helped to identify key principles for the design of an effecti9ve FPRT process; these included the need to be underpinned by a culture of collaboration, be voluntary, be confidential and foster critical reflection (Atkinson & Bolt 2010; Chester 2012; Hendry, Bell & Thomson 2014; Gosling 2014). The design embedded a structure for reviewees to engage in critical reflection and collaborative dialogue with peers using the review of teaching criteria, before and after peer review.  Such processes of critical reflection and creation of an environment of trust and collaborative ownership are in accordance with current scholarship (Chester 2012; Gosling 2014).  Principles of Educational Design Thinking: (Brown 2008; Golden et al 2011) including Empathising, Defining, Ideation, Prototype development and Testing, facilitated a robust design process. Firstly, the needs of various groups of university teachers were carefully considered, through discussion with colleagues, and consulting with key educational leaders who constituted the project’s advisory team. This process led to the clear definition of the requirements and the development of new ideas to achieve these. A prototype of the process and FPRT reviewer training package were developed and trialled within internal groups to seek feedback and refined as needed. Learnings from each iteration informed further refinements to finally produce a process and a training package that has been widely acclaimed for their effectiveness.  

Key outcomes 

1. The development of a reviewer training program was a key outcome. This training program included strategies to develop deeper understanding of the peer-review process and the criteria (dimensions), with calibration activities that enabled discussion of what constitutes good practice in relation to each dimension. This was a valuable activity in itself, as this led to many in-depth discussions amongst academics from various disciplines around the meaning of teaching quality. The wide interest generated by the project led to the training of 80 reviewers from four faculties at UNSW.  

2. A webpage for the project that includes background information relating to the project and the process was set up. This provided a template for other faculties, and a number of websites have been established by other faculties: https://teaching.unsw.edu.au/summative-peer-review  

3. The project led to the development of recommendations to improve the process of peer review. These recommendations have been submitted to the PVC(E) office and it is expected that these would inform refinement of the overall process at UNSW.  

4. Preliminary findings from qualitative component of research showed both reviewers and reviewees found the review process to be “an enjoyable experience and useful in terms of [their] own learning”. As one reviewee noted, it was a “privileged viewpoint that you don’t normally get”. Reviewees found the process promoted reflective practice in teaching; while reviewers comments highlighted that doing the observation acted as a “trigger” for new ways of engaging students, especially students from diverse backgrounds.  An important aspect of the FRPT process as noted by a reviewee was helping them to “prepare for this formal process of summative peer review”.  An additional and important finding was that reviewers found the process very beneficial – they commented on the many benefits of observing a colleague in a different context and the value of observing the impact of teaching strategies. More details are available at this link to the full report. 

5. Published manuscript: Harrison, R., Meyer, L, Rawstorne, P, Razee, H., Balasooriya, C. (2020). Assessing and enhancing quality in higher education teaching practice: A meta review. Studies in Higher Education https://doi.org/10.1080/03075079.2020.1730315  

6. Conference abstract accepted: Meyer, L, Rawstorne, P, Razee, H., Harrison, R., Balasooriya, C. Piloting formative peer review of teaching: strengthening academic professional development. (Accepted to HERDSA 2020, delayed to 2021).  

7. Workshop conducted in collaboration with Iain Skinner: Australasian Association for Engineering Education conference (AAEE 2019). 

8. International symposia and workshops: Invited symposia at the Asian Medical Education conference (AMEA 2019), invited workshops at the Colombo Conference on Medical Education 2019, Invited workshop for clinical educators at the University of Peradeniya, Sri Lanka, invited workshop for clinical educators at Medical University 2 (UM2), Myanmar. 

The main challenges that impact on the process of conducting a FPRT process relate to perceived time commitments (which are in part related to the anxiety of preparing for reviews). The workload involved in conducting reviews, the challenges of scheduling reviews (both in terms of time availability and scheduling systems/admin support), and the concerns expressed by a few reviewers around critiquing their colleagues, needs further attention. 

Next steps 

Research component: Data that is collected via the reviewer training sessions is being analysed to assess the eight-dimension structure of the FPRT instrument. The qualitative data are being analysed to better understand the experience of engaging in the FPRT process from the perspective of the reviewers and reviewees.   

From a practical perspective, the further expansion of the formative peer-review process across UNSW is a high priority. There is room to adapt the process to suit the contexts of each discipline and/or faculty. Feedback also suggests that the scope of the formative process may need to extend beyond the delivery (teaching) to include the review of design and content. There has also been a suggestion to possibly incorporate student perceptions alongside peer observations.   

Key References 

• Gosling, D. (2014). Collaborative peer-supported review of teaching. In J. Sachs & M. Parsell (Eds.), Peer Review of Learning and Teaching in 
Higher Education (pp. 13–31). Netherlands: Springer. doi: 10.1007/978-94-007-7639-5  

• Yiend,J., Weller,S. & Kitchin,I (2012). Peer observation of teaching: the interaction between peer review and developmental models of practice. Journal of Further and Higher Education, Retrieved online: http://www.tandfonline.com/doi/abs/10.1080/0309877X.2012.726967#.U0VEKJhOVMs 

• Atkinson, D., & Bolt, S. (2010). Using teaching observations to reflect upon and improve teaching practice in higher education. Journal of the Scholarship of Teaching and Learning, 10(1-19).  

• Chester, A. (2012). Peer partnerships in teaching: Developing a voluntary model of professional development in tertiary education. Journal of Scholarship in Teaching and Learning, 12(2), 94-108  

• Hendry, G.D. & Oliver, G.R. (2012). Seeing is believing: The benefits of peer observation. Journal of University Teaching & Learning Practice, 9(1). Retrieved from http://ro.uow.edu.au/jutlp/vol9/iss1/7 

1. Balasooriya, C., Harrison, R., Rawstorne, P., Meyer, L., Razee, H. (2017-2018). Designing, piloting and evaluating a process for formative peer-review of teaching ($45,000). UNSW Scientia Education Investment Fund 
2. Velan, G., Steel, A., Lim, B., Sherry, C., Thomson, C., Balasooriya, C., . . . McIntyre, S. (2017-2018). Development and implementation of an education portfolio to evaluate excellence of teachers at UNSW ($50,000): UNSW Scientia Education Investment Fund.
3. Torda, A., Damodaran, A., Balasooriya, C., Fenton-Kerr, T., Mobbs, S., & O'Sullivan, P. A. (2018). The CLASSIE project – Clinically applicable student studies in Ethics, for Phase 3 medical students ($ 43,455): UNSW Scientia Education Investment Fund.
4. Balasooriya, C., MacIntyre, R., Travaglia, J. (2015-2017). Adapting and trialling an innovative international approach to address burnout in medical students ($20,000). University of New South Wales/SJTU-UNSW Collaborative Research Fund - Seed Grant 
5. Balasooriya, C., Canalese, R., Shulruf, B., Zwar, N. (2014). Enhancing the use of Hypertension Guidelines in General Practice ($75,000). National Heart Foundation of Australia – Vanguard Grant (Category 1 - ACGR)
6. Balasooriya, C., Olupeliyawa, A., O’Sullivan, A., Hughes, C., Rhee, J. Development of a mobile application to facilitate learning and assessment of teamwork skills in clinical settings. ($65,000). UNSW 2014 Allocation of Recurrent HWA Funding
7. Balasooriya, C., (2012). Clinical decision making in health care settings ($5000). SPHCM Small Grant to Support the development of Category 1 Grant application
8. Rutherford, A., Thompson, R., Balasooriya, C. (2010/2011). Enhancing the content, delivery and assessment of Graduate Capability 2 in the MBBS program ($6000): University of New South Wales Strategic Learning & Teaching Development Fund.
9. Balasooriya, C., Taylor, S., & Koritschoner, E. (2010/2011). Assessment and development of Teamwork across all Phases of UNSW Medicine Program ($10,000): University of New South Wales Strategic Learning & Teaching Development Fund.
10. Balasooriya, C D., Hughes, C. (2009). Assessing teamwork in undergraduate medical education to better prepare students for teamwork requirements of interns ($4000): UNSW Seed Funding for ALTC Learning & Teaching Grant Applications.
11. Harris, P., Bloomfield, L., Hughes, C., Balasooriya, C., Meyer, L. (2007). Evaluation of assessment methods and training related to curricula for postgraduate training in diagnostic radiology and radiation oncology. Royal Australian and New Zealand College of Radiologists

Peer Reviewed Educational Publications

1. Chunming, W. M., Harrison, R., MacIntyre, R., Travaglia, J., & Balasooriya, C. (2017). Burnout in medical students: a systematic review of experiences in Chinese medical schools. BMC Medical Education, 17(1), 217. doi: 10.1186/s12909-017-1064-3
2. Iqbal, M., Velan, G. M., O’Sullivan, A. J., & Balasooriya, C. (2016). Differential impact of student behaviours on group interaction and collaborative learning: medical students’ and tutors’ perspectives. BMC Medical Education 16:217. DOI: 10.1186/s12909-016-0730-1 
3. Dharmaratne, S., Razee, H., Ponnamperuma, G., Marambe, K., Dawson, A., & Balasooriya, C. (2015). Vignette-based Decision Point Analysis (VDPA): A novel method of exploring contextual influences on clinical decision-making. European Journal for Person Centred Healthcare. 3(2): p.151-157
4. Jiayu, M., Xianming, K., Wang, C., MacIntyre, R., Travaglia, J., Balasooriya, C. (2015). Comparative study on medical education training models in China and Australia. Chinese Journal of Medical Education Research 14 (1).
5. Balasooriya, C., Rhee, J., Shulruf, B., Canalese, R., & Zwar, N. (2014). Evaluating the Guideline Enhancement Tool (GET): an innovative clinical training tool to enhance the use of hypertension guidelines in general practice. BMC Medical Education, 14(1): p.1-5. 
6. Olupeliyawa, A. M., O’Sullivan, A., Hughes, C., & Balasooriya, C. D. (2014). The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): A Workplace-Based Assessment Focusing on Collaborative Competencies in Health Care. Academic Medicine. 89 (2)
7. Olupeliyawa, A. M., Balasooriya, C. D., O’Sullivan, A., & Hughes, C., (2014). Educational impact of an assessment of medical students’ collaboration in healthcare teams. Medical Education. 48(2), 146-156. doi: 10.1111/medu.12318
8. Balasooriya, C., Olupeliyawa, A., Iqbal, M., Lawley, C., Cohn, A., Ma, D., et al. (2013). A student-led Process to Enhance the Learning and Teaching of Teamwork Skills in Medicine. Education for Health. 26(2), 78-84. doi: 10.4103/1357-6283.120698
9. Olupeliyawa, A. M., O’Sullivan, A., Hughes, C., & Balasooriya, C. D. (2013). Transition to clinical practice as a medical graduate: what collaborative competencies and behaviours are critical? Focus on Health Professional Education. 14(2): 57-70
10. O'Sullivan, A J. , Harris, P., Hughes, C., Toohey, S.M., Balasooriya, C., Velan, G., Kumar, R.K. and McNeil, H. P. (2012) Linking assessment to undergraduate student capabilities through portfolio examination, Assessment & Evaluation in Higher Education, 37(3): 379-391. 
11. Balasooriya, C., Tetik, C., & Harris, P. (2011). Why is my design not working? The role of student factors. Research Papers in Education. 26(2), 191 - 206.
12. Almeida, P., Teixeira-Dias, J. J., Martinho, M., & Balasooriya, C. D. (2011). The interplay between students' perceptions of context and approaches to learning. Research Papers in Education, 26(2),149-169.
13. Balasooriya, C., Di Corpo, S., and Hawkins, N. (2010). The facilitation of collaborative learning: what works? Higher Education Management and Policy, 22(2), OECD publishing, © OECD
14. Albergaria-Almeida, P., Teixeira-Dias, J. J., Martinho, M. & Balasooriya, C. (2010). Kolb’s learning styles and approaches to learning: the case of Chemistry undergraduates with better grades. International Journal of Knowledge Society Research, 1(3), 1-16.
15. Balasooriya, C. D., Toohey, S & Hughes, C. (2009). The Cross-Over phenomenon: Unexpected patterns of change of Students’ Approaches to Learning. Studies in Higher Education 34(7),781-794.
16. Balasooriya, C., Hughes, C. & Toohey, S. (2009). Impact of a new integrated medicine program on students’ approaches to learning. Higher Education Research & Development 28(3), 289-302.
17. Olupeliyawa, A. M., Hughes, C., & Balasooriya, C. D. (2009). A review of the literature on teamwork competencies in healthcare practice and training: Implications for undergraduate medical education. South East Asian Journal of Medical Education, 3 (2), 61-72.
18. Balasooriya, C., Toohey, S. & Hughes, C. (2005) What do Deep & Surface Approaches to Learning mean for medical students & medical education? Focus on Health Professional Education,6(3), 29-42
19. Senanayake, M. P., Ranasinghe, A., & Balasuriya, C. (1998). Street Children - a preliminary study. Ceylon Medical Journal, 43, 191-193.

Invited Commentaries and Other Journal Contributions

1. Olupeliyawa, A., & Balasooriya, C. (2015). The impact of programmatic assessment on student learning: what can the students tell us? Medical Education, 49(5), 453-456. doi: 10.1111/medu.12730
2. Iqbal, M. P., O’Sullivan, A. J., Velan, G., & Balasooriya, C. (2014). Are we seeing the full picture of peer-assessment in medical education? Medical Teacher, 36(11), 1007-1007. doi: 10.3109/0142159X.2014.940880
3. Koritschoner, E., Balasooriya, C., Bloomfield, L. (2009). Clinical Skills centres don’t have to cost an arm and a leg. Letter. Medical Teacher. 31: 873-4.

Book Chapters

1. Balasooriya, C., Asante A., Jayasinha R., Razee H. (2014). Academic mobility and migration: Reflections of International Academics in Australia. In Tight, M. and Maadad, N. (Eds.), International Perspectives on Higher Education Research. Emerald Press.
2. Balasooriya, C., Tetik, C., & Harris, P. (2013). Why is my design not working? The role of student Factors. In Evans C, Kozhevnikova M. (eds.), Styles of Practice in Higher Education: Exploring approaches to teaching and learning. Routledge
3. Macdougall, C., Ritchie, JE., Rotem, A. & Balasooriya, CD. (2003). Educational Options to Develop Population Health Skills in the Primary Health Care Workforce. Thinking Populations: Population Health and the Primary Health Care Workforce. Public Health Education and Research Program. Australian Primary Health Care Research Institute.
4. Boyle, F., Donald, M., Rotem, A., Balasooriya, C., & Ritchie, J. (2003). Future Directions for Information Dissemination: Population Health and the Primary Health Care Workforce. Thinking Populations: Population Health and the Primary Health Care Workforce. Public Health Education and Research Program. Australian Primary Health Care Research Institute.

Peer Reviewed Conference Papers and Posters 

1. Shamim, S., Balasooriya, C., Torda, A., Baig, L., Zubairi, N. (2018) Evaluation of workbook as an educational strategy for ethics in undergraduate medical education: a qualitative case study approach. Oral presentation at Ottawa-ICME 2018, Abu Dhabi, United Arab Emirates.
2. Balasooriya, C., Rhee, J., Olupeliyawa, A., Zwar, N., Shulruf, B., Canalese, R., & Jabbour, V. (2016). The experience of using the Scrip Concordance Test in educational research. Poster presented at the Ottawa 2016 and ANZAHPE 2016 Conference, Perth, Australia. 
3. Balasooriya, C., J. Rhee, and A. Olupeliyawa. The Decision Enhancement Tool (DET) in ANZAHPE - OTTAWA 2016 Joint Conference 2016. Perth, Western Australia.
4. Pervaz Iqbal, M., O'Sullivan, A. J., Velan, G. M. & Balasooriya, C. (2015). The Collaborative Learning Development Exercise (CLeD-EX) tool: A Preliminary Investigation. Paper presented at the AMEE 2015 conference Glasgow, Scotland
5. Pervaz Iqbal, M., Balasooriya, C., O'Sullivan, A. J., & Velan, G. M. (2014, 7 - 10 July 2014). Identifying the critical and important collaborative competencies that relate to effective collaborative learning environments in medicine. Paper presented at the ANZAHPE 2014 Conference, Gold Coast, Australia.
6. Dharmaratne, S., Balasooriya, C., Razee, H., Marambe, K., & Ponnamperuma, G. (2014). Vignette-based Decision Point Analysis: An innovative method to explore sensitive clinical practice information. Paper presented at the Asia Pacific Medical Education Conference, Singapore.
7. Balasooriya, C., Hughes, C., O’Sullivan, A., Taylor, S., Koritschoner, E., Olupeliyawa, A., Harris, P., & Tancred, E. (2011). An Exploration of the Benefits and Challenges of using a Student-Led Approach to Enhance Medical Students’ Development of Teamwork Skills. ‘Personally Arranged Learning Session’ presented at the Australian & New Zealand Association for Health Professional Educators ANZAHPE 2011 conference, Alice Springs, Australia.
8. Balasooriya, C., Tetik, C., & Harris, P. (2010). New Insights into the context-dependent nature of learning approaches. Paper presented at the ELSIN 2010 "Exploring styles to enhance learning and teaching in diverse contexts" conference, Aveiro, Portugal.
9. Almeida, P., Teixeira-Dias, J., Martinho, M., & Balasooriya, C. (2010). Kolb’s learning styles and approaches to learning: the case of Chemistry undergraduates with better grades. Paper presented at the ELSIN 2010 "Exploring styles to enhance learning and teaching in diverse contexts" conference, Aveiro, Portugal.
10. Balasooriya, C. D., Di Corpo, S., & Hawlins, N. (2009). Facilitation of small group learning: What works? Paper presented at the OECD Quality of Teaching in Higher Education conference, Istanbul, 09. 
11. Harris P., Balasooriya C.D., Titek, C. (2008). What were the examiners thinking? Using post-hoc analysis of oral examiner deliberations to clarify standards. Paper presented at the AMEE conference, September 2008. 
12. Harris P.,  O’Sullivan A.J., Toohey S.M., Hughes C.S., Velan G., Balasooriya C.D., Kumar R.K., & McNeil H.P. (2008) A Method for Assessment to Drive Learning toward Desired Graduate Capabilities. Paper presented at the 13th Ottawa International Conference on Clinical Competence (Ozzawa08) 
13. Koritschoner, E., Barclay, A. & Balasooriya, C. (2008) Campus-based teaching of clinical skills in the early years of an integrated undergraduate medical program. Paper presented at the 13th Ottawa International Conference on Clinical Competence (Ozzawa 2008).
14. O’Sullivan, A., Toohey, S., Velan, G., Harris, P., Hughes, C., Ham, J., Reid, S., Balasooriya, C. & Kumar, K. (2007) An aligned assessment scheme in a medicine program. Paper presented at the ISSOTL conference, Sydney, Australia. 2-5 July.
15. Olupeliyawa, A.M., Balasooriya, C.D. & Hughes C. (2010) Facilitating the transition from medical student to intern: how can the assessment of clinical collaborative competencies help? Association for Medical Education in Europe (AMEE) conference, from 4 to 8 September 2010, Glasgow, UK, book of abstracts, p.431
16. Olupeliyawa, A.M., Hughes, C. & Balasooriya, C.D. (2010) Teamwork roles and competencies for a junior medical officer: their context in clinical practice and in outcome-based curricula. ANZAME Conference, from 14 to 16 July 2010, James Cook University, Australia, book of abstracts, p.196
17. Olupeliyawa, A.M., Balasooriya, C.D. & Hughes, C. (2010) Teamwork competencies in healthcare for undergraduate medical education: Understanding the context. 7th Asia Pacific Medical Education Conference, from 4 to 8 February 2010, National University of Singapore. book of abstracts p.293
18. Olupeliyawa, A.M., Balasooriya, C.D. & Hughes, C. (2010) Developing teamwork as an outcome in an outcome-based medical education program. 7th Asia Pacific Medical Education Conference, from 4 to 8 February 2010, National University of Singapore. book of abstracts p.292
19. Balasuriya, C., Dharmawardena, D, Rajapakse, L, & Jayasinghe, S. (2000a). Approaches to learning and examination performance. Paper presented at the Sri Lanka Medical Association 113th Academic Sessions, Colombo, Sri Lanka.
20. Jiffry, M. T. M., Fernando, S, Balasooriya, C D & Thenuwara, W. (2000b). A preliminary study on the relationship between performance at GCE(A/L) and MBBS examinations. Paper presented at the Annual Sessions, Sri Lanka Association for the Advancement of Science 2000, Sri Lanka.Wijesuriya, M., &  Balasooriya, C D. (1998). Summary  of  work  -  National Diabetes Centre. Paper presented at the Sri Lanka Medical Association 111th Academic Sessions, Colombo, Sri Lanka.

School level contributions:

• School of Public Health and Community Medicine Academic Quality Committee

Faculty level contributions:

• Medicine, Assessment Development and Evaluation Committee
• Medicine, Program Evaluations and Improvement Group
• Medicine, MedTELT Committee
• Medicine, Inspired Learning Initiative Committee
• Medicine, Phase 1 Committee

UNSW level contributions:

• UNSW Scientia Education Academy
• AAUT Citation Selection Committee
• Program Design & Delivery, Delivery Subject Matter Group

External contributions:

• Australia and New Zealand Association for Health Professional Education, President (2018 - present), Committee of Management and Fellow