Elisa Baniassad, associate professor of teaching in the Faculty of Computer Science, explores using ChatGPT to create not only quiz questions, but the associated practice questions.
by Elisa Baniassad
ChatGPT can be a useful tool for designing test questions using prompts. You can provide those same prompts to your students to generate new questions (or variants of the same questions) that they can use to practice. I decided to try this out to see how well it would work.
Note: The example in this article is very small and constrained. You can try this with a more sophisticated prompt, and change settings to allow for more variation in generated questions.
This website features stories and content submitted by individuals within the UBC community. We value the diversity of perspectives and experiences that these stories bring to our platform. Any views or opinions expressed in these user-generated stories are solely those of the individual authors and do not necessarily reflect the views or opinions of UBC as a whole. The presence of user-generated content on our website does not imply endorsement or agreement by UBC with any particular perspective.
Step 1: Make my quiz question using a prompt
Here is what I prompted:
Ask me a question that will test my knowledge of the difference between the Single Responsibility Principle and the Interface Segregation Principle at a level that doesn't require coding knowledge.
(Note: I am keeping this example high level on purpose, for easier understanding of this post)
For context, these are principles in Software Engineering that guide where functionality is placed within code. I asked ChatGPT to come up with an explainer for each one:
- The Single Responsibility Principle means that each thing should do just one job, so if you need to make a change, you only have to change that one thing.
- The Interface Segregation Principle means that when you want to use something, you shouldn’t be forced to use parts you don’t need, so it’s better to have smaller, separate pieces that you can choose from.
ChatGPT’s question:
Sure! Here's a question to test your knowledge of the difference between the Single Responsibility Principle (SRP) and the Interface Segregation Principle (ISP): Question: Imagine you have a toolbox with various tools, including a hammer, a screwdriver, and a wrench. How does the idea of "single responsibility" apply to the toolbox, and how does the "interface segregation" concept relate to how you use these tools? A) The single responsibility of the toolbox means that each tool should only serve one specific purpose, while interface segregation means that you can pick and choose which tools to use for a particular task. B) The single responsibility of the toolbox means that each tool should be able to perform multiple functions, while interface segregation means that all tools in the toolbox must be used together for any task. C) The single responsibility of the toolbox means that it should only contain one type of tool, while interface segregation means that the tools should be interchangeable with each other. D) The single responsibility of the toolbox means that you should use only one tool for all tasks, while interface segregation means that you should use all the tools at once for any job. Choose the correct option that best represents the concepts of SRP and ISP in the context of the toolbox analogy.
The correct answer is A and C. This is a pretty good exam question to test basic non-technical understanding, and the answer is correct.
I would now rewrite and save this answer, and ask it on my quiz. The nice thing is that I can even use the answer guide and explanations from ChatGPT if I want to provide more rationale to students when releasing grades. Rewriting and modification are needed because there is a risk with all ChatGPT-generated content that it has copied directly from another source without citing it. A good practice is to use ChatGPT as an inspiration as opposed to using its text outright.
Step 2: Give the prompt to my students so they can generate practice questions
I can now give the same prompt to my students, so they can use it in ChatGPT. The nice thing is that the questions it asks them will be original. Since it is a pretty specific topic, the variants for this might seem fairly close; they are different enough that it would feel like a different question to the student.
Pasting the same prompt into another chat window we get this new question:
Question: Imagine you have a class called "Vehicle" that handles both movement and fuel management. According to the principles of SRP and ISP, is this a good design? Why or why not? A) Yes, it's a good design because the Vehicle class handles multiple responsibilities efficiently. B) No, it violates SRP because a class should have only one reason to change, and it has two (movement and fuel management). C) No, it violates ISP because clients might be forced to depend on both functionalities even if they only need one of them. Select the correct options: B only C only Both B and C
This time the correct options are B and C. Once again, this is a pretty good assessment question, at the level requested. It is not far off something I may have provided students myself for them to use for practice.
The student can submit an answer, and ChatGPT will tell them if it is right or wrong. This approach can be superior to a practice test with a rubric because it is dynamic: ChatGPT will even give an explanation for why they are wrong. The student can ask for clarification, and can even ask “If I thought this was right, what am I not understanding?” ChatGPT will try to figure out where the student’s gap in understanding lies!
Below is an example of “where am I going wrong with my thinking” from another question and answer session:
This is an insightful response to the student! I like this pedagogically investigative opportunity for students using ChatGPT.
Then the student can use the prompt again, and they will get another version of the question. They can keep trying and asking for clarification about why they are not answering correctly. They can keep going until they get the right answer every time!
Why I (personally) am enthusiastic about this
I spend tens of hours designing assessments, and even more tens of hours making associated practice problems because of how robust they have to be, and the fact that they require explanations in accompaniment. Being able to generate both will free up a lot of time. I can shift that time to doing other things, like talking to students individually, or even doing one-on-one grading should the need arise!
No, I haven’t tried this in flight yet
I haven’t had a chance to try this out on real students yet! However, given the quality of the questions ChatGPT generates and the expertise it shows in the answers, I’m confident enough to give it a go. I will report back with how it goes.
One thing I will need to figure out well is how to provide an accessible option. I will likely create a long chat session with a link so that students can view a whole conversational interaction without having to use the tool themselves. Of course, we always have office hours as a real-life option for going over course materials.
Learn more about prompting
You can read more about sophisticated prompts by replaying our recent Prompathon, and learning about different kinds of prompts.