Suggestions on how to encourage students to document use of GenAI for various assessment-related tasks.
As teachers and course designers, two key decisions need to be made regarding the use of generative AI:
- Do students still need to learn how to complete a given task without any use of AI? (For assurance of learning purposes, such assessments should occur in an environment where AI cannot be used, e.g. an oral assessment.)
- Where collaboration with AI is allowed, how should students document its use?
9 aspects of assessment with suggested documentation
Understanding the task requirements
Possible use of AI and example tools
- Summarising the task instructions (Bing Chat)
- Translating the task instructions into a student’s first language (Google Lens or Google Translate)
- Explaining how the marking criteria relate to the task instructions
- Creating an assessment checklist based on the task instructions and marking criteria (ChatGPT)
Documenting
For academic integrity purposes, there’s probably no need for students to document this part of the process, assuming that nothing generated in this step is used in the final submission.
You could still encourage them to share any prompts and tools they used for this step, along with a discussion/reflection on its usefulness.
Researching/Finding information
Possible use of AI and example tools
- Suggesting key texts or researchers relevant to the topic (ChatGPT)
- Finding sources related to a key text (Elicit)
- Using information generated by AI as a source
Documenting
As above, the main purpose for documenting this step would be to encourage students to reflect on the research process and/or provide them with feedback on the effectiveness of their chosen search strategy.
The use of information generated by AI as a source of information would likely be discouraged, unless the aim is for students to identify possible inaccuracies or biases in the generated text.
Taking and synthesising notes from different sources
Possible use of AI and example tools
- Summarising sources (ResearchRabbit)
- Synthesising sources (Elicit)
Documenting
This is a contentious area. You probably want your students to read texts for themselves and learn to identify and evaluate key information. However, students will almost certainly use AI tools to summarise and critique texts.
Students could be asked to document tools they used to summarise or analyse source texts, and explain how they attempted to verify the accuracy of the summaries.
Analysing data
Possible use of AI and example tools
- Analysing quantitative data sets (ChatGPT4 with plugins such as Wolfram)
- Identifying themes or patterns in qualitative data (Any LLM-based AI such as ChatGPT)
Documenting
This will be a common use of GenAI in the workplace. Guide students on how to do it ethically and effectively. If they need to know how to do a certain type of analysis without AI, this needs to be assessed in an environment where AI cannot be used.
If we allow AI to be used in the analytics process, students should be required to:
- document which tools they chose to use and why
- explain how they used each tool
- explain how they tried to verify the accuracy of the analysis
- reflect on how useful they found the process, and whether they would do anything differently in future
Brainstorming/Ideating
Possible use of AI and example tools
- Suggesting research questions/topics
- Suggesting ideas for products, projects, processes etc.
- Giving feedback on ideas
Documenting
This is likely to be one of the major uses of AI in most workplaces. We don’t currently expect students to acknowledge discussions with peers that might have helped shape their ideas, so it’s probably not necessary to require them to document use of AI for this purpose, unless we wanted students to reflect on their metacognitive process.
Planning and structuring
Possible use of AI and example tools
- Listing the required steps to successfully complete the task
- Proposing a structure for the given genre (ChatGPT)
- Providing bullet points of relevant information to include in each section
Documenting
Clear guidelines should be given to students regarding which (if any) of these steps they can use GenAI to assist with.
If students are allowed to use AI to suggest a detailed structure, they should be required to include an appendix outlining the prompts they used and the responses generated.
Critical analysis
Possible use of AI and example tools
- Evaluating the arguments of others and the evidence on which they are based
- Interpreting and evaluating evidence from your own research
- Inserting your own voice into your writing to analyse and critique concepts, situations, theories or events
- Developing a coherent and evidence-based argument or theory
- Using examples, case studies, analogies or metaphor to explain ideas in meaningful ways
Documenting
Students still need to develop their critical thinking and evaluate situations, processes, actions etc. However, technologies will be used to assist with analysis and evaluation, so assessments will need clear guidelines. Include examples to show whether/how students can use AI for this aspect of a task. If students are being assessed on what they added or amended to an AI generated analysis, this needs to be made clear in the task instructions and the marking criteria.
Documentation would likely require full transcripts of all prompts and their responses, with the student’s amendments or additions to AI-generated text clearly highlighted or explained.
Citation and referencing
Possible use of AI and example tools
- Citations/references are already included in texts that have been generated by AI (Chat)
- Sources are suggested by AI tools and full referencing details are provided by these tools (Elicit, ResearchRabbit)
- Students’ reference lists are submitted to AI for editing as required
Documenting
Given that texts are increasingly likely to be initially generated by AI, with the student then adding and amending text as required, a key part of learning and assessing will be based around students demonstrating their process, especially with regard to the verification of information.
Essentially, students will need to provide a satisfactory answer to the question: what did you do to satisfy yourself that the information you have provided is accurate, relevant, sufficient, evidence-based, and inclusive of multiple perspectives?
If students submit a written text as part of an assessment, consider:
- requiring students to submit two separate documents – a text generated by AI that responds to the assignment question, and a separate text where the student details the process they followed, and the sources they consulted, to verify the text
- a document that uses colour-coding (plus accessible indicator) to demonstrate AI-generated text and additions/amendments by students (including supporting citations)
- an annotated reference list noting aspects such as which of the listed sources were part of an AI-generated text, how students verified if sources cited by AI were accurate and relevant, which sources the students found of their own accord, and how they used those sources to improve the original text
Mechanics of language (spelling, grammar, syntax, cohesion)
Possible use of AI and example tools
- Generating a full text that is grammatically and syntactically correct (any LLM-based technology)
- Translating a text the student has written in their own language into grammatically correct English (Google Translate)
- Prompting students to use certain words and phrases, or make changes to spelling, grammar or word choice, while the student is writing a draft (Grammarly, MS Word)
- Editing a completed draft that a student has written to ensure that the grammar and spelling are correct, and that the text is cohesive (ChatGPT)
Documenting
Decision-making in this instance will depend on professional and disciplinary requirements, as well as the learning outcomes of the course, subject and task.
Wholly AI-generated text needs to be clearly documented as such. But indicating collaboration at the level of spelling, grammar or word choice will be impractical and probably not useful. Learning outcomes and marking criteria will need to be rewritten to reflect the changing context.
It needs to be made very clear to students whether/how they can use AI. If we don’t want students to use such tools at all, we need to a) explain why not, and b) be honest with ourselves that the only way we can be assured that a student has written a text is if they write it in class.
For the purpose of most texts, it seems more useful to require students to acknowledge when/how they have used AI through the form of appendices with transcripts, and/or to document their process through a written reflection.
