With the rapid rush to move to online teaching prompted by the pandemic, you may be wondering how to run online exams effectively. To be effective in an online environment, assessments that worked in a face-to-face classroom may need to be tweaked or even replaced.
Is it possible to deliver open book online exams while also ensuring academic integrity without a lot of supervision? How can we support flexibility for students and protect their privacy at the same time?
Education Focussed (EF) academics Dr Nirmani Wijenayake, Lecturer and Acting Deputy Director of Teaching in the School of Biotechnology and Biomolecular Sciences tells us how.
What has been your experience with running online exams?
My course on Fundamentals of Biochemistry (BIOC2181) had digital assessments (worth 25% each) for a large cohort of 240 students. In July and August 2019, I used the Safe Exam Browser (SEB) for delivering two open book digital assessments.
SEB is a web browser environment to carry out e-assessments safely. For my digital assessments, I had created Moodle quizzes beforehand, and set them up with password protection on SEB.
What is the main premise for using open book exams online? And is there a particular challenge with open book testing in online environments?
Many of my students try to learn biochemistry through rote learning rather than actually understanding the material. One of the main reasons for this is that this is a very content-rich subject and there is a lot to learn in a short amount of time. Most of them get bogged down by the details rather than trying to understand the bigger picture and the relevance of what they are learning. Students often overestimate how much detail they actually need to learn for exams.
By making the exam open book it takes some of the stress away from the students so they can focus more on what's important. They are also compelled to answer in more critical and analytical ways thus encouraging high-order thinking skills.
I personally don't see the sense in students trying to learn enzyme names or details of pathways by memorizing them because this information is at their fingertips and they can easily look it up.
The biggest challenge with an open book online exam is that students may copy the questions or try to reach out for help through social media. But this can be combated through technology like the SEB.
What appealed to you about using this technology for your teaching practice?
For me it wasn’t about the technology. I was already planning to run my exams online. I just reached out to the Digital Assessment team to enquire about tips on running online exams and they asked me if I would like to trial the SEB system. I agreed because I thought it might be interesting and I like trying new things.
What are the top 5 things you recommend someone keep in mind before introducing a fully online open-book exam?
- Make sure that students understand that open book doesn't mean they will automatically do better. This is a common misconception they have. They still need to know their content well coming into the exam because it is still a timed exam and they won't have time to look up everything.
- Write good questions where they need to understand the material to be able to answer the question. You may also need to have a larger question bank depending on the size of the course.
- Plan well and in advance. Running an online exam is not something you can decide a week before the exam.
- Be willing to deal with technology issues and be flexible if something goes wrong. It is important to have a backup plan in case technology fails on you.
- If you are conducting an online exam for the first time, I recommend consulting someone who is more experienced to guide you through the process. Mathew Hillier and Naveen Kumar from the PVCE Digital Assessment team helped me immensely with this aspect.
With the move to online teaching, some teachers may be tempted to simply convert a classroom-based assessment into an open book digital format – as a quick fix. What challenges do you see with that approach?
Changing a current assessment to an open book theory exam is not a good way to go forward, because, a good assessment tests students in aspects other than just disciplinary knowledge such as their ability to do research, communicate, teamwork, problem-solve, etc. You will be taking away their learning opportunities if you do that. It would be better to try to find alternate ways to do the same thing online.
We should still aim at meeting the same quality learning standards as before. Otherwise students will feel that we don’t care about them.
I think there are a lot of avenues available to us for guidance - through EF Communities of Practice such as the Online Learning and Innovation CoP and EF Student Feedback and Digital Assessment CoP, other workshops, expert tutorials, resource repositories. You would be able to find something that works for you.
If you are time-poor, just reach out and ask other EF colleagues. The Faculty of Science has also created a Science Online Learning Support channel on Teams for all the Science schools - anyone can reach out for help.
Online assessments spark concerns about academic integrity. What are your thoughts about this?
This wasn’t an issue for me because we ran these exams in our labs invigilated in 2019. The students were only allowed to access the quiz and pre-approved whitelisted sites through the SEB, so they couldn’t get any help from others online either. But bear in mind that SEB does not work for remote learning.
I also run a very large fully online general education course and academic integrity is an issue in this course because there is no way for me to know if the student sitting a quiz is actually the student enrolled in the course.
I think it would be extremely beneficial for us if the University invested in having an online exam centre where academics can book them for exams. The focus shouldn’t be about being able to do online exams from home, but being able to do them effectively from class rooms.
What has been the feedback from the students? And would you trial fully online open book exams again?
As I mentioned earlier, my goal with fully online open book assessments, is to measure my student’s achievement of the course learning outcomes, instead of simple memorisation.
In that sense, the student feedback as seen below has been extremely positive. I definitely plan to run all my exams online now. I don’t see myself going back to the paper versions.
Tell us more about the student experience
My students had positive things to say about the entire experience -
"The open book exam, allowed me to learn the content rather than stressing about memorising. As long as I knew how all the pathways worked and all the information mattered, it made the final exam extremely easy to navigate."
"I also liked the way we applied our knowledge in tests. to do well, we still needed a good understanding of the course content. I thought this was practical."
The biggest issue for the student and staff is the reliability of technology. When we surveyed the students before they did online exams, their biggest concern was whether technology was going to hold up. And it did become an issue in the second test as Moodle went through an upgrade the day before the exam and made the whole system extremely slow on the day of the exam which stressed both staff and students. However, despite that students were still supportive of online exams.