Designing Online Assessments in the Age of AI

Cari Coetzee

Cari Coetzee

AI Solutions Architect

Designing Online Assessments in the Age of AI

Table of Contents

Generative AI is transforming how students learn, research, write, collaborate, and demonstrate understanding. This is forcing higher education institutions to rethink what meaningful assessment looks like online. In this article, we explore 10 practical strategies for designing more effective online assessments in the age of AI and introduce some of the emerging frameworks helping institutions navigate this rapidly evolving landscape. 


 

Generative AI has fundamentally changed the assessment conversation in higher education.  

When students can instantly generate essays, summaries, code, research support, and even study guides using tools like ChatGPT and Claude, traditional online assessments no longer provide the same visibility into student learning that they once did. It is one of the reasons why OES is spearheading our Assurance of Learning framework globally. 

It’s important to bear in mind that AI does not signal the end of academic integrity or meaningful assessment. However, it does signal the need for smarter assessment design. 

Institutions need to redesign assessments to emphasize critical thinking, encourage application and reflection, foster collaboration, and make the learning process more visible and valued, all of which add up to a more authentic demonstration of learning. 

So here are 10 practical strategies for designing online assessments that remain rigorous, engaging, and meaningful in the age of AI:

1. Start with assessment blueprinting and AI intentions

Effective assessment design begins with clarity. 

Before creating an assessment, educators should define:• The intended learning outcomes: Be clear about what students should achieve. • The skills students should demonstrate: Focus on the capabilities you want to assess. • The evidence of learning you want to see: Decide how students will demonstrate understanding. • The role, if any, that AI could (or should) play: Set expectations for appropriate AI use.They also need to set their intentions to: 

• Preserve meaningful cognitive work: Ensure students do the thinking that matters. • Make learning visible: Assess the process, not just the final product. • Avoid harmful cognitive offloading: Use AI to support learning, not replace it.

In the AI era, assessment blueprinting must go beyond content mastery alone. Institutions are increasingly required to think carefully about when AI use is appropriate and when it is not. And those expectations must be communicated to students and other stakeholders. 

Some institutions are beginning to adopt structured approaches for defining acceptable levels of AI involvement in assessment tasks. For example, they may allow AI usage in brainstorming or collaborative drafting. 

By clarifying expectations early, educators can reduce ambiguity while creating more authentic learning experiences.

2. Design questions AI cannot easily answer superficially 

The increased usage of and access to generative AI means that simple fact-recall or generic essay prompts may no longer provide meaningful insight into student understanding. 

Instead, effective online assessments should prioritize: 

• Contextual application: Apply knowledge to authentic situations. • Critical analysis: Evaluate evidence and justify decisions. • Reflection: Explain how thinking has evolved. • Interpretation: Draw conclusions, not just summarize. • Lived experience: Connect learning to real contexts. • Problem-solving: Tackle realistic challenges. • Discipline-specific judgment: Demonstrate professional thinking.AI can generate information quickly, but it’s far less effective at replicating nuanced human reasoning, personal reflection, contextual awareness, and applied professional thinking. 

For example, rather than asking students to summarize a leadership theory, educators might ask them to apply that theory to a workplace challenge they have personally experienced, critique a real-world case study, or explain how their thinking evolved over time. 

These types of assessments make learning more authentic while encouraging students to engage more deeply with course material.

3. Diversify assessment methods beyond traditional essays

One of the most important shifts happening in higher education is the move away from overreliance on a single assessment format, and with good reason! In the age of AI, diverse assessment methods are becoming increasingly valuable because they allow educators to evaluate learning from multiple angles. 

Consider incorporating: 

• Video reflections: Assess understanding in students' own words. • Presentations: Combine communication with subject knowledge. • Collaborative projects: Develop teamwork and shared problem-solving. • Discussion-based assessments: Encourage dialogue and critical thinking. • Simulations: Apply learning in realistic scenarios. • Peer critiques: Build analytical and feedback skills. • Iterative drafts: Reward learning through revision. • Real-world problem-solving tasks: Reflect workplace practice.Different assessment formats reveal different dimensions of student learning. They also better reflect modern professional environments, where success rarely depends on producing a single written output in isolation.

4. Build transparency into every assessment

Many academic integrity challenges emerge from ambiguity rather than misconduct, particularly as institutions grapple with policy and governance around AI usage. The antidote is clarity. 

Students need clear guidance around: 

• Assessment expectations: Explain what success looks like. • Success criteria: Make standards transparent. • Grading standards: Show how work will be evaluated. • Acceptable AI use: Clarify where AI is appropriate. • Collaboration boundaries: Define when students should work together. In the AI era, transparency matters more than ever. Emerging frameworks such as Transparency in Learning and Teaching (TILT) help institutions rethink how purpose, process, and evaluation criteria are communicated throughout the learning journey. 

Providing clear rubrics, examples, and instructions not only improves student confidence, but also supports equity and consistency across online learning environments. 

When students understand what meaningful success looks like, they are far more likely to engage authentically with the learning process. 

5. Responsibly use AI as a learning tool

Much of the early conversation around AI in education focused on plagiarism detection and academic misconduct. 

But the conversation is rapidly shifting toward a more important question: 

How can AI support learning? 

Some institutions are now using AI to create: 

• Adaptive formative feedback: Deliver more personalized guidance. • Role-play simulations: Practise real-world conversations and decisions. • Personalized tutoring support: Reinforce learning on demand. • Scenario-based activities: Encourage authentic application. • Guided practice environments: Build confidence through repetition.These approaches allow students to practice skills and engage with course concepts in more dynamic ways. They also expedite the feedback loop to give students more frequent and relevant guidance.  

Throughout, educators remain central to the process. Human judgment, facilitation, mentorship, and feedback continue to play a critical role in ensuring meaningful learning outcomes.

6. Incorporate real-world and workplace-relevant scenarios

One of the most effective ways to strengthen online assessments is to make them feel authentic. 

Assessments that mirror real-world challenges encourage students to apply knowledge in practical, meaningful contexts while developing workplace-relevant skills. 

For example: 

• Business students: Analyze current organizational challenges. • Healthcare students: Respond to virtual patient scenarios. • Education students: Design classroom interventions. • Sustainability students: Balance competing stakeholder priorities. The question is no longer whether graduates will use AI professionally. The question is whether institutions are preparing them to use it critically, ethically, and effectively. 

In many professions, AI is already becoming part of everyday workflows. Authentic assessment should therefore move beyond simply measuring what students know and begin evaluating how they apply that knowledge, exercise judgment, verify information, and make decisions when AI tools are available. 

The rise of AI makes authentic assessment more important, not less. If students will use AI in the workplace, assessments should reflect that reality by evaluating how effectively they can work with AI, question its outputs, identify its limitations, and apply human judgment in context. 

Well-designed real-world scenarios create opportunities for students to demonstrate precisely those capabilities while making learning more engaging, relevant, and meaningful.

7. Require reflection and process visibility 

In an AI-enabled environment, understanding how students arrived at an answer can be just as important as the answer itself. To this end, reflective and process-oriented assessment practices are becoming increasingly valuable. 

Consider incorporating: 

• Reasoning processes: Show how conclusions were reached. • Explanations of choices: Justify key decisions. • Justifications of decisions: Assess judgment as well as outcomes. • Critiques of AI outputs: Evaluate rather than accept AI responses. • Contextual applications: Apply learning to authentic settings. • Performance or demonstrations: Assess skills in action. • Iterative improvements: Show growth over time. • Annotated drafts or logs: Capture the learning journey. • Oral defence or viva: Test understanding through discussion. • Commentary on AI use: Explain how AI supported the work. • Initial attempts before AI-supported revision: Make progress visible. • Evidence of refining AI suggestions: Demonstrate critical evaluation. • Verification notes: Show how information was checked. • Cross-checking against disciplinary standards: Validate work using trusted sources. • Revision logs or change summaries: Record meaningful improvements. • Reflections on AI limitations: Encourage responsible AI use.These elements encourage students to demonstrate their decision-making and intellectual growth throughout the assessment process. They also help educators better understand student learning while reducing overreliance on polished final outputs alone. 

In many ways, the AI era is pushing higher education toward a stronger emphasis on learning processes rather than purely finished products.

8. Foster peer assessment and collaboration 

Online learning can sometimes feel isolating. Collaborative assessment practices help create stronger learning communities while developing essential human skills. 

Peer assessment activities encourage students to: 

• Evaluate ideas critically: Strengthen analytical thinking. • Communicate constructively: Practise giving useful feedback. • Engage with diverse perspectives: Learn from others' viewpoints. • Reflect on quality and standards: Deepen understanding of assessment criteria. • Learn through dialogue: Reinforce learning through discussion.In an AI-enabled environment, uniquely human capabilities such as collaboration, empathy, communication, and judgment become even more valuable. Digital discussion forums, peer-review tools, collaborative documents, and group simulations are all useful methods to support meaningful interaction in online courses. 

The goal is not simply to assess individual performance, but to cultivate deeper engagement and shared learning.

9. Prioritize frequent, formative assessment and feedback

Traditional high-stakes assessments often provide feedback too late to meaningfully support learning.  

Frequent and formative assessments create opportunities for students to receive more immediate guidance and reflect on it throughout the learning process. 

These low-stakes activities might include: 

• Short quizzes: Check understanding early. • Discussion prompts: Encourage regular engagement. • Draft submissions: Enable feedback before final assessment. • Knowledge checks: Monitor progress throughout the course. • Reflective exercises: Connect learning to experience. • Peer feedback activities: Learn by reviewing others' work.AI is also accelerating interest in adaptive formative assessment models that provide students with more personalized, real-time feedback throughout the learning process, which is something that has traditionally been difficult to achieve at scale in higher education. 

When implemented effectively, formative assessment shifts learning into an ongoing developmental process.

10. Focus on growth, judgment, and human insight

As AI tools become more capable of generating content, the uniquely human dimensions of teaching become even more important. 

Meaningful feedback should go beyond grading alone. It should help students: 

• Understand strengths and weaknesses: Identify areas for improvement. • Improve their reasoning: Develop stronger critical thinking. • Deepen their understanding: Move beyond surface learning. • Build confidence: Reinforce progress and capability. • Develop professional judgment: Prepare for real-world decision-making.Audio and video feedback, coaching conversations, and personalized commentary can all help strengthen instructor presence in online learning environments. 

Students do not simply need a score or more information. They need guidance, interpretation, encouragement, and thoughtful challenges. 

AI may assist with aspects of assessment and feedback delivery, but human insight remains central to transformative learning experiences. 

Designing assessments for the AI era requires a new playbook 

As generative AI reshapes higher education, institutions are being challenged to rethink not only academic integrity, but also transparency, feedback, adaptability, and authentic learning design. 

Frameworks exploring adaptive learning, ethical AI integration, formative assessment, and transparent teaching practices are beginning to influence how universities approach online learning at scale. 

OES explores these ideas in greater depth in our resource, Leveraging AI for Transparent and Adaptive Learning, which examines how institutions can integrate AI into teaching and assessment responsibly, ethically, and effectively. 


Sources 

Hansen-Brown, A., Sullivan, S., Jacobson, B., Holt, B., & Donovan, S. (2022). College students’ belonging and loneliness in the context of remote online classes during the COVID-19 pandemic. Online Learning, 26(4), 323–346. https://doi.org/10.24059/olj.v26i4.3123

Share:

Keep Reading for More Insights and Inspiration!

Skip to content