Make the process visible

§01 Black Box Assessment

Assessment as a black box

We send out a task and get a product back. How it was made stays hidden. Naomi Winstone, Karen Gravett and Sam Elkington (2026) call this the black box of assessment, and they argue that prevention and control can keep the problem going, because they keep everyone’s attention on the finished product.

Their name for the alternative plays on the flight recorder in aviation, which is kept as an archive for learning, not for punishment. For a piece of student work, the flight data can be drafts, research logs, reflective notes and AI chats, and once the thinking is visible in them, integrity grows without anyone checking the product harder. The name carries a second idea too. Errors, false starts and changes of direction should be brought into the open and credited.

Four principles

Four building blocks

A black box assessment has the task itself, at least one learning outcome about the process, windows into the box, and criteria that credit the quality of the process: revision, reflection and how problems were solved. The product still counts. You have to decide carefully how much weight it gets against the process. The authors suggest two or three windows that mix naturally occurring traces with short reflections, fit the discipline and are ideally chosen with the students.

Two prompts make change visible: What did you believe at the start that you later changed your mind about, and why? and Which feedback or new knowledge changed your thinking most?

§02 Activity

Windows into the box

Choose two or three windows that could fit your assessment. Each card says what the window lets you see, what it costs, and how easily it could be produced afterwards.

Four objections, four answers

Not everyone has the same room to fail. Errors have different consequences for different students, and students notice when a teacher praises mistakes while the system punishes them. The article is a design proposal. It reports no empirical study, and its effects on learning and workload are still untested.

§03 A real problem

Authentic in practice, or only on paper?

A task that is authentic only on paper, such as an invented case, protects little. AI easily makes up plausible local examples, and in a British study examiners couldn’t tell authentic assessments with and without AI contributions apart (Kofinas et al., 2025). The authors recommend assessment that builds on social and experiential learning and happens in real time, so that it doesn’t rest on written products alone.

Authentic practice is something else. When students work over time on a real problem, collect their own data, build and test prototypes and deal with people in the field, the work leaves traces: in the field, with clients and in supervision. End the project with an oral defence of the process, and the assessment draws on all four kinds of evidence from Step 1. That makes it one of the most robust forms we have, even if AI helps a lot along the way.

AI can also let students get further, perhaps past the prototype to something that is actually used. When that happens, the criteria should reward the quality of their choices and their reading of the situation, however much they end up producing.

§04 Activity

Fix the AI answer

One design move is to give students an AI-generated answer in their subject and ask them to assess it, correct it and justify the corrections with sources. It takes knowledge of the field and checking, which is the point. Try it yourself on a paragraph about the research in this course. Click every sentence you think is wrong, then check.