A durable, adaptable, descriptive approach to assessment in an AI-influenced era.
One major result of the Generative AI and Assessment research project has been the development of an assessment design framework based on assessment submissions and follow-up interviews with participants.
The 4C Assessment Design (4CAD) framework has emerged from an identified set of instructor approaches to addressing the increasing availability of generative AI and its use by students in various aspects of their learning.
These approaches are captured by four Cs:
-
Collaboration. The integration of generative AI as a working partner in the completion of the assessment. This approach tends to stress adaptability and interactivity with the technology as a means of demonstrating learning outcome(s) the assessment is meant to measure.
-
Critique. The analysis and critical evaluation of AI-generated content, often accompanied by a reflection on the human–AI interaction. This approach tends to focus on the relevance, accuracy, utility, and overall value of AI outputs and the use of AI in the process of completing the assessment (or parts thereof).
-
Content Disclosure. The documentation of generative AI use. This approach tends to emphasize transparency around the inclusion of AI-generated content as well as the different ways in which AI is incorporated into the process of completing the assessment (or parts thereof).
-
Confirmation. The verification of student work. This approach tends to seek both compliance with policies at the course, program, and institutional levels and assurance that the student can demonstrate intended learning outcomes without the use of AI.
Putting the Pieces Together
Using these categories to describe and classify assessments offers something closer to a typology of approaches, rather than a scale or hierarchy. Furthermore, with 4CAD, we have also intentionally avoided classifying or evaluating task-based use cases for generative in assessment (e.g., AI is “good” or “bad” for doing X), anticipating that AI development and capability will likely outpace any such claims.
We chose to visualize the framework with an unconventional Venn diagram, showcasing the various interplay and overlap among the four functions.
Figure 1. The numbers on the figure correspond to a unique ID associated with each assessment submission. The areas with asterisks (*) and hyphens (-) indicate overlapping categories for which we did not receive any relevant assessment submissions. In the discussion that follows, we consider the possible features of assessments that might hypothetically fall into these categories.
Table 1. For each ID on the Venn diagram, the following table provides a short assessment description and a link (if available) to the full assessment text.
| ID | Assessment Description |
|---|---|
| #10 | Video Game Covers with Firefly |
| #13 | Generative AI and ChatGPT: What Do They Know? |
| #35 | Completing Incomplete Program Evaluation |
| #43 | In-Class Poetry Analysis |
| #68 | AI Use Document |
| #73 | AI Literature Project |
| #78 | Critiquing Multiple Choice Questions |
| #82 | ChatGPT Balanced Scorecard |
| #94 | Online Writing Session |
| #102 | Advanced Editing with AI Prompt Engineering |
| #164 | First-Year Drama Essay |
| —/*** | No assessments submitted (speculative) |
A Few Highlights
In our analysis, we found that most assessments instructors submitted fell into two or more categories. If an assessment suggested using AI as a collaborator on a specific task, it would often pair the collaborative function with some form of critique: at the very least, fact checking and reviewing any AI output before integrating it into the deliverable.
There are, however, some key exceptions or anomalies worth noting.
First, some assessments did, in fact, incorporate the use of AI for collaboration or critique exclusively, rather than a mix of both, and a smaller number of assessments that allowed AI use did not require any disclosure of use.
Second, at the time of our research (September 2023–April 2024), confirmation was largely viewed in terms of verifying student completion of an assessment without AI assistance. Hence, there were no assessments that allowed for the use of AI and also implemented some form of confirmation. To clarify, we understand confirmation as an instructor-imposed method of determining how the assessment was completed, rather than a student-centred method (i.e., content disclosure, which assumes the student is being honest about how they used AI). Invigilated assessment represents a form of confirmation with which most would be familiar. Some instructors mentioned returning to in-class assignments, but we also saw confirmation in the form of oral exams or viva-style assessments following the submission of a written deliverable.
Since conducting this research, tools like Grammarly’s Authorship allow for confirmation to be applied to assessments that permit the use of AI by tracking and identifying human vs AI-generated writing throughout a submission. Instructors using technology like this could design an assessment that would fall into the area of the Venn diagram marked with hyphens (-): an assessment, for example, that tracks student use of AI in the writing process to ensure that it is being used in specific ways (brainstorming, creating an outline, identifying gaps, challenging arguments, etc.).
The other area of the Venn diagram to which we were unable to map any of the submitted assessments was the overlap between collaboration and critique without any content disclosure, indicated by the area with asterisks (*). One possible explanation for the lack of assessments in this space is that if an instructor is asking students to use AI in several different ways, they would likely want students to disclose and/or reflect upon these use cases.
What’s to Come
This research has more or less been paused since collecting and analyzing the assessments (winter 2024), conducting the follow-up interviews (winter/spring 2024), and finalizing the interview transcriptions (summer 2024). Next steps include a deeper reading of the interview transcripts within the context of the 4CAD framework and a final, published manuscript. These steps have been on hold largely due to the Principal Investigator (Ben Lee Taylor) cutting a postdoctoral fellowship short to accept a full-time writing specialist role with McMaster University’s Writing and Academic Skills team.
Despite the delay, however, the 4CAD framework has proven useful to instructors, educational developers, and others trying to grapple with and make sense of generative AI’s impact on teaching and learning in higher education.
Inquiries can be directed to Ben Lee Taylor.