Published 18 September 2023
AI case study: Minimising GenAI usage through simulated role-play
The academics & their views on GenAI
Andrea Benvenuti is an Associate Professor of Politics and International Relations. His approach to learning and teaching is based on Project-Based Learning (PBL), which encourages students to learn by experiencing and solving real-life problems.
Andrea sees generative AI as an impressive technology that students will increasingly utilise, particularly as it becomes more sophisticated. He believes we should work with this technology. At the same time, he also understands GPT raises some significant issues in the context of his specific discipline, namely the likelihood that students will use it to generate unoriginal responses.
Assessment overview
| Course |
ARTS3783 Great, Emerging and Declining Powers in the Contemporary World |
| Assessment type |
From traditional essay to a Foreign policy review & simulated role-play of a National Security Council (NSC) meeting |
| Goal |
1. In-depth consideration of the threats, challenges and problems facing policymakers in relation to the international issues with a geopolitical, military, economic dimension.
2. The application of critical and evaluative thinking to influence/guide decision making and policy options by the NSC. |
| Assessment focus |
1. To develop a concise and persuasive foreign policy review document.
2. To discuss, advocate and defend a specific policy or course of action in a simulated role-play. |
Driving change factor
One of the reasons why Andrea moved away from the traditional essay was to minimise student plagiarism. More fundamentally, he wanted to expose students to the challenges faced by real-life policymakers in formulating a coherent foreign policy towards a country, region or multilateral political issue. In doing so, he wanted to enhance student understanding of complex real-world problems, develop their critical thinking and communication skills, and nurture their creativity. The simulated role-play of a National Security Council meeting meant that students engaged in a novel and stimulating assessment with the application of evaluative and critical thinking within a hypothetical case scenario.
GenAI response reliability
The simulated role-play format discourages students from using GenAI tools as they created poorly formed and generic responses with incoherent writing. There was also the challenge of creating effective prompts to address the assessment criteria.
Strategies that minimised Generative AI responses
The assessment consists of 2 parts, a written foreign policy review and simulated role-play based on a real-world foreign policy crisis scenario. The complexity of the scenario and specificity of the discipline made it difficult for generative AI to produce sound responses while the role-play requiring live student negotiation and input further reduced reliance on generative AI.
- Student live presentation of their position according to their assigned stakeholder viewpoint
- Debate style role-play to defend recommendations, counter-argue, and identify potential areas of compromise and/or agreement
- Real-time negotiation and decision-making required during the role-play simulation.
- Problem-solving and development of recommendations based on a real-life scenario
- A real-life case scenario is a crisis that incorporates tensions, conflicting priorities, and situations requiring a deep level of analysis and evaluative thinking based on subject expert knowledge.
- In-depth research of current political events/issues & ramifications to inform viable policy options based on sound rationale.
What did students learn?
| Benefits |
Key points:
- the policy review and mock cabinet meetings offer a refreshing alternative to traditional essays assessment, motivating students to approach scenarios practically and tackle them from a real-world perspective
- felt positively challenged from an analytical and communicative perspective, critical thinking is fostered
|
| Challenges |
The challenge in this assessment task relate to the limited knowledge exposure of how governments work for some students along with the specific strategic, diplomatic, economic, military and geopolitical issues occurring in the world. Hence, there was some unfamiliarity with conducting a foreign policy review and engaging in a debate style role-play to address the assessment criteria.
|
| Solution |
Student scaffolding and a template were provided to assist students and a clear focus on issues and activities that supported student learning to complete this assessment was embedded throughout the course term.
|
Student views about the assessment
"I enjoyed the structure of the policy review in comparison to a classic essay. I was more engaged in writing as a political actor rather than a non involved academic (as a traditional essay would)."
"Not only did writing the policy reviews provide us with very useful skills in policy writing and analytical thinking, but it also allowed us a great deal of creativity and the opportunity to apply theory in 'real world' scenarios."
"The observations and lessons that emerged from doing the policy reviews have been instrumental in shaping my strategic mindset over the course of my career. They are practical, constructive, and worthwhile...My perspective on this has only strengthened since transitioning into the workforce.”
"It afforded me with an empowered sense of autonomy as a student, and a researcher, to explain my own judgments and conclusions to advocate for a desired outcome. hte interplay between historical interpretations and modern assumptions in academia was also very fascinating, making the policy reviews a far from ordinary task."
"We enjoyed how they contributed to other exercises in the courses, including policy simulations and mock cabinet meetings. From my experience, the policy reviews seemed to motivate us to take things a bit more seriously and engage more with the course."
Enjoyed this article? Share it with your network!
