Assessment Design using Generative AI

GenAI and Assessment Design

  • The rapid advancement of Generative AI in achieving high scores on standardized exams like the LSAT and GRE, as well as in producing written assignments that sometimes surpass those of students, has underscored an urgent need in higher education to devise assignments that uphold academic integrity and rigor. Furthermore, it is important for faculty to recognize the emerging skills associated with using Generative AI and to explore opportunities to incorporate these models into their teaching practices.

    In response to this swiftly changing environment, faculty members have begun to revise their courses and, in some cases, the entire curriculum to either curb the use of Generative AI or to appropriately incorporate it into various assignments and assessments. The following resources outline strategies for revising and creating assignments and assessments, and provide examples of how UBC faculty have been addressing these challenges.

    The GENAI Assessment Scale

    The GENAI Assessment Scale (Perkins et al., 2024) is an useful resource for educators, designed to define and communicate acceptable uses of Generative AI (GenAI) in academic contexts. This scale supports academic integrity through six specified levels:

    .

    No AI: This level ensures rigorous and equitable assessment by prohibiting AI, ideal for testing knowledge and comprehension. This level is suitable for testing knowledge and comprehension, for example, a traditional multiple-choice exam or an in-class essay written without the use of AI tools. 

    AI as a Study Tool: Students can use Generative AI to prepare, review, and study. AI tools serve as mentors or study buddies, suitable for drafting assessments or study guides. This level is suitable for students seeking personalized learning experiences and efficient study aids. AI tools can help students understand complex concepts and improve retention. They can also facilitate the creation of comprehensive study materials, and serve as catalysts for collaborative learning, encouraging peer-to-peer interaction and knowledge sharing among students. 

    AI for Idea Generation: AI tools can be used in the initial stages for brainstorming and idea generation, but not in the final submission. This level is suitable for assessments requiring students to demonstrate their writing skills, such as constructing their own essays. For example, students might use AI tools to help generate ideas for a research paper, or a persuasive essay. In this approach, AI can assist in overcoming a student’s block, expanding on initial thoughts, and providing diverse perspectives or topics for consideration. However, the students are responsible for developing, refining, and presenting their own arguments, analysis and conclusions. 

    AI-Assisted Editing: AI tools can be used to outline responses or organize notes, focusing on the final quality of writing, word choice, and expression. This level is ideal when the assessment focuses on the quality of the ideas and understanding, independent of the quality of language and expression. Students might use AI tools to receive instant feedback on their draft essays or identify areas of improvement in their writing. AI can assist in refining grammar, punctuation, style, and coherence, ensuring that the student’s message is conveyed clearly and effectively. In this approach, students are encouraged to develop their own ideas and structure their arguments while using AI to enhance readability and polish their work. 

    AI Output Evaluated: Students use AI to generate content and critically evaluate its accuracy, bias, and citations, fostering productive learning and evaluative judgment. This level is suitable for assessments that require students to discern accuracy, identify biases, and assess the reliability of sources through examination of citations. This level also fosters a deeper level of engagement with course material, promotes independent learning, and cultivates a heightened awareness of the nuances involved in evaluating information in a GenAI era. 

    Full AI: AI is used as a ‘co-pilot’ to enhance creativity and meet assessment requirements.This level is suitable when the outcome of the assessment is judged on the earlier organization, idea generation, discussion, orals, and other methods. For example, students might use AI to create a comprehensive summary of a group discussion or synthesize research findings into a cohesive report. AI can act as an assistant, helping students organize their thoughts, integrate various viewpoints, and present their findings in a coherent and creative manner. In this level, AI tools might assist in drafting outlines, suggesting structure, and even providing initial content that students can then build upon and refine.

    Each level targets specific ways of integrating and communicating allowed use of Generative AI to students

    Detecting the Use of GenAI

    The current capabilities of Generative AI are presenting many faculty with the challenge of continuing to ensure that their assessments, assignments, courses, and programs are equitable, promote learning, and overall maintain academic integrity. Currently, GenAI detectors are not accurate, and there are limitations in the ability for humans to detect GenAI use. However, by redesigning our assignments to include in-class elements, direct connections to local and current issues, and a focus on the process in addition to the product, we can make GenAI use easier to detect and mitigate when it is not appropriate for learning goals.

    For more information, the Academic Integrity site at UBC is a helpful resource.

How we can reduce the risk of GenAI doing the work for students

  • While the use of GenAI by students may sometimes not support their learning, it can also be a valuable tool that helps students develop new skills and approaches in our discipline. Additionally, these tools can be leveraged to create more personalized and engaging learning experiences.

    We hear from our students that they use GenAI when they are feeling stressed and anxious about their grades. 

    Include In-class Elements

    Bringing assessment or elements of assessments into the classroom can help reduce the risk of students using Generative AI in their submissions.

    • In-person exams, such as midterms and final exams  limit AI misuse.
    • In-person presentations and interviews allow for real-time assessment reducing AI use.
    • Including in-person elements as part of larger assignments ensures active participation and reduces AI-driven submissions. 
    • Graded group work, peer-review, Q&As, debates, and other active learning approaches foster collaboration and limit AI usage.
    • Case-based and problem-based learning engage students in real-world scenarios, and minimize AI-generated responses.

    Emphasize Process rather than Product 

    Making the process more visible helps ensure the authenticity of student work and reduces the likelihood of AI misuse.

    • Deemphasizing the weight of the final product by including multiple steps such as outlining, drafting, peer-reviewing, and accepting/rejecting feedback emphasizes the process over the outcome.
    • Providing ways for students to explain how they came up with a result or answer ensures transparency and reduces reliance on AI.
    • When working in teams, students co-create a team contract to promote accountability and minimize the use of AI-generated work.
    • Post-submission interviews allow instructors to verify the student’s understanding and reduce AI-related concerns.

    Design Multimodal Assignments

    Utilizing various formats for assignments enhances engagement and reduces reliance on generative AI, as current AI tools do not effectively generate multimodal outputs.

    • In addition to traditional research paper-style writing, students may demonstrate their learning through videos, podcasts, infographics, etc.
    • Students annotate videos or other media and are evaluated based on their annotations, ensuring active engagement and reducing AI dependency.

    Emphasize Tasks That AI Cannot Perform Well

    Educators have been developing alternative assessments that ChatGPT cannot easily complete. This approach carries risks, given the rapid evolution of technology and the continuous addition of massive amounts of prompts, which improve the outputs generated. However, while making assessments completely AI-proof may be nearly impossible, we can focus on tasks in assessment expectations that foster personalized and localized connections. For example, demonstrating learning could involve linking to local contexts or current events, which may not be well represented in large language models.

    Update Grading Schemes

    You might consider updating your current grading schemes, rubrics, and outlined criteria to prioritize and emphasize competencies that are less mechanical and harder for AI to replicate. For instance, in a writing assignment, you could shift the focus towards skills like constructing solid arguments, incorporating and assessing evidence, and conducting critical analysis, rather than emphasizing grammar and essay structure. Additionally, asking students to submit rough planning notes with their assignments could help evaluate their process, not just the final product.

How to make the most of GenAI in assessments and assignments

  • While there are times when the use of Generative AI (GenAI) by students may not support their learning, it can also be a valuable tool that helps them develop new skills and approaches within our discipline. Additionally, these tools can be leveraged to create more personalized and engaging learning experiences.

    Effective Practices for Integrating GenAI In Assessment and Assignment Design

    Protect Student Privacy: Generative AI (GenAI) currently poses considerable privacy risks. To mitigate these risks, GenAI assignments should ensure that students only use GenAI tools with a completed Privacy Impact Assessment (PIA) and avoid inputting personal information. For a more detailed description of privacy considerations, please refer to the UBC GenAI privacy guidelines.

    Communicate How and When Students Can Use GenAI: In order to preserve academic integrity and develop clear guidelines for assignment completion, help students develop an understanding of which assignments they can use GenAI for and how they can use the model. 

    Scaffold the Use of GenAI to Ensure Student Learning and Equity: Students have varying levels of proficiency and confidence in using GenAI. To help students develop their skills and ensure a level playing field, it is essential to scaffold and support their use of GenAI for assignments.

    Promote Ethical and Responsible AI Use in Assignments: There are considerable ethical issues with the use of Generative AI and students are not always aware of these. Support students in understanding challenges including intellectual property, accuracy and bias when using these tools.

    Emphasize Human-AI Interaction: Use AI as a tool to augment, not replace, human creativity. Assign tasks where AI can help brainstorm ideas, generate drafts, or provide inspiration, but require students to refine and personalize the outputs.

    Emphasize ways for students to demonstrate their learning process. This could range from showing their work when writing an equation to reflecting on developing a project and what steps they used. 

    Include Reflective Components:To help students develop GenAI literacy and enhance their learning, provide opportunities for them to reflect on and evaluate GenAI outputs.

    Align with the Learning Goals: It is important to keep the focus on the intended learning goals; do not get distracted by the emergence of GenAI tools. Remember what you expect your students to learn and how each assignment may guide their learning. Communicate your passion and the meaning behind each assignment. 

    Ways of Integrating GenAI in Assignments and Assessments

    Students Evaluate and Revise GenAI Output 

    As of 2024, Generative AI (GenAI) output may frequently include inaccuracies, hallucinations, and biases. Despite these limitations, they can be effectively utilized to enhance student learning. By generating and analyzing GenAI outputs, students can identify errors, biases, and different perspectives, and then work on revising these outputs to either improve them or reflect on the errors detected. This practice not only leverages GenAI limitations but also aids in developing evaluative judgment—an essential component of critical thinking, especially important given the growing reliance on GenAI.

    UBC Examples

    Anubhav Pratap-Singh: FNH 303

    Using ChatGPT to suggest formulas and replacement ingredients for food product recipes, in an undergraduate courseIn FNH 303 (Food Product Development), Dr. Pratap-Singh has modified the traditional “Product Selection” part of the assignment for New Product Development, allowing and encouraging students to use ChatGPT to identify recipes (ingredients and their compositions) for new food product development. Students are expected to use the Internet to modify an existing product in the market to either: a) make it more sustainable by removing problematic ingredients/processes; or b) modify the product to either add a claim (for example gluten-free; plant-based; etc.). Dr. Pratap-Singh is encouraging students to seek replacement ingredients using ChatGPT, as well as use other internet sources to describe known chemical and physical properties of different ingredients in the food product under consideration. To demonstrate their understanding of how each ingredient works, students are expected to address how modifications of the formulations may lead to food products with different traits (sweeter, chewier, etc).

    Jared Taylor MICB 418: Industrial Microbiology

    In MICB 418: Industrial Microbiology, Jared Taylor encourages students to responsibly utilize generative AI tools for their final group presentations on ethical and legal issues within industrial microbiology. Students must critically evaluate and verify the accuracy of the information provided by these AI tools, and they are required to document their usage of the tools throughout their project development. To support this, Dr. Taylor provides an academic integrity checklist and a reflective survey, developed with Dr. Simon Bates, which students must complete and submit. This process not only ensures adherence to academic standards but also promotes deep reflection on the role and reliability of AI in scientific research and presentation.

    Kari Grain, ADHE 328: Social Institutions of Adult Learning

    In a course on contemporary changes in adult education, students use ChatGPT to analyze recent developments in the field. Based on Chuang’s 2021 article, which highlights areas such as the growth of technology-based learning, increased cross-cultural training, and a stronger focus on adult learner characteristics, students are tasked with exploring these themes. They access ChatGPT and input a personalized prompt asking for specific ways they can learn best as adults, considering their preferred method of learning (e.g., listening, playing, doing, imagining). Students then paste ChatGPT’s response into their discussion posts and evaluate its usefulness and accuracy, while also reflecting on potential issues with relying on AI for research-based information. This exercise not only enhances their understanding of adult learning but also demonstrates the practical application of AI tools in educational settings.

    Vikas Menghwani, SUST 200: Application, Practice & Management Approaches

    In a course on contemporary changes in adult education, students use ChatGPT to analyze recent developments in the field. Based on Chuang’s 2021 article, which highlights areas such as the growth of technology-based learning, increased cross-cultural training, and a stronger focus on adult learner characteristics, students are tasked with exploring these themes. They access ChatGPT and input a personalized prompt asking for specific ways they can learn best as adults, considering their preferred method of learning (e.g., listening, playing, doing, imagining). Students then paste ChatGPT’s response into their discussion posts and evaluate its usefulness and accuracy, while also reflecting on potential issues with relying on AI for research-based information. This exercise not only enhances their understanding of adult learning but also demonstrates the practical application of AI tools in educational settings.

    Brainstorming, outlining and Ideation 

    Generative AI is a useful tool for ideation, outlining and brainstorming and faculty are increasingly incorporating GenAI as part of the thinking or writing process. This can involve students using GenAI to brainstorm a list of topics, create an outline for a project, generate ideas for a persuasive essay, or use it to draft aspects of their writing. 

    One of the primary ways GenAI is being utilized is to assist students in brainstorming and generating ideas. GenAI can be prompted with a topic or concept, and it can generate a list of related ideas, topics, or questions to explore. This can help students overcome writer’s block and expand their thinking, reasoning beyond their initial thoughts. GenAI can also be used to create mind maps or visual representations of ideas, aiding in the organization and structuring of thoughts.

    In addition to brainstorming, GenAI is being used to draft aspects of students’ writing. For example, students can use GenAI to generate topic sentences, outlines, or even complete paragraphs. This can save time and provide a starting point for students who may struggle with getting their thoughts on paper. GenAI can also be used to generate different versions of a sentence or paragraph, allowing students to compare and select the best option.

    UBC Examples

    Kari Grain, ADHE 328: Social Institutions of Adult Learning

    In a creative exercise inspired by Stivers and Lehrman (2023), students explored the future of adult education through AI-generated art on Padlet. They envisioned their ideal learning environments, considering key aspects of adult learning discussed in class. After describing their visions on Padlet, the AI generated six versions of their environments, from which students selected their favorite. They then explained the symbolism and connection to course readings. This exercise demonstrated the creative use of AI in education, allowing students to engage with the material in an innovative way while exploring the potential of AI tools in teaching and learning.

    Students Incorporate GenAI as An Assistant throughout the Assignment

    In some teaching contexts, faculty are encouraging students to use Generative AI throughout the assignment process without placing any restrictions on its use. While students are required to disclose when and how they have used Generative AI, they are encouraged to integrate it into their assignments. This approach is applied across different disciplines and is often used in courses that assess students’ coding skills.

    UBC Examples

    Qingshi Tu, Course Title

    As part of the introductory Python lecture, students were tasked with creating a Python script to simulate the 649 lottery. They were encouraged to utilize ChatGPT to enhance their learning experience by: (1) asking it to explain the mechanism of the 649 lottery, and (2) generating the Python code required for the simulation. Students then tested the generated code in Google Colab, using ChatGPT further to debug and refine their scripts. This exercise not only helped students understand the lottery mechanism and Python programming but also demonstrated the practical use of AI tools in coding and debugging.