Sabie Systems' interviews for its new AI roles were not working, and everyone involved could feel it. Panels asked candidates to compare model families and name the latest releases, which mostly tested who had read the most news that week. The take-home exercise produced near-identical, polished submissions, because almost every candidate used an AI assistant to complete it. When panels met to decide, they argued from impressions, and the most confident panellist usually won.
None of that told Sabie whether a candidate could do the job. And when an unsuccessful candidate asked for feedback, the honest answer would have been "we are not entirely sure".
CloudNala's view is that an interview for an AI-era role should test the verbs in the role profile with work samples, state its rule on AI use in advance, score against written anchors before the panel discusses, and keep a record of why the decision was made. That is fairer to candidates, more useful to the organisation, and easier to defend.
This article is general information about designing interviews, not legal or HR advice. Employment law, including the Employment Equity Act's rules on assessments, applies to how you test candidates; take advice from your HR and legal people before changing a selection process.
This is the last part of the series. Part 1 decoded roles into verbs and work products, Part 2 rewrote existing roles around outcomes, and Part 3 built learning paths. Hiring is where all three come together.
Sabie Systems is fictional, and its systems and figures are illustrative. Sabie is a Pretoria software and data services company of about 140 people, currently hiring for five AI-related roles, including an evaluation-focused role for its customer support assistant.
Why the old interview breaks
Two familiar interview formats stopped working at the same time.
The take-home exercise now measures access to AI. If a candidate can complete it with an assistant in an hour, the output says little about them. Banning AI from a take-home is close to impossible to enforce, and in most AI-era roles it would also be testing the wrong thing, because using AI well is part of the job.
Knowledge questions now measure memory of volatile facts. Which model has the largest context window, or what a particular framework calls its agents, changes every few months. A candidate who knows today's answers has read recently. That is not nothing, but it is not the job.
What the job requires is visible in the role profile from Part 2: core verbs, decisions owned and work products. The interview should test those directly.
Start from the verbs in the role profile
The method has five steps, and each traces back to the role profile.
Start with the role's core verbs, usually two or three. A role with six core verbs is not a hiring problem; it is a design problem, and Part 1 explains why.
Design one short work sample per core verb, based on realistic work. Short means 30 to 60 minutes, done during the interview rather than as unpaid homework.
Observe against a written list of what the panel is watching for. Decide this before the interview, not afterwards.
Score independently against anchors, before any discussion.
Decide as a panel, comparing scores and reasons, and record the decision.
Design a work sample for each verb
Good work samples are small, realistic and observable. Sabie uses these patterns, adapted per role.
Frame. Give the candidate a messy request, such as a short email thread from a client asking for "an AI that answers our customers", and ask for a one-page problem statement. Watch whether they name the user, the outcome, the boundary and the questions they would ask first.
Build or integrate. Give the candidate a small pull request produced by a coding agent that connects an assistant to an internal system. Ask them to review it aloud. Watch whether they notice the over-broad permission, the missing failure handling and the untested path.
Evaluate. Give the candidate twenty answers from a support assistant, the product documentation they should be grounded in, and a simple rubric. Ask them to find the failures and say what pattern they see. This is Sabie's core sample for its evaluation role.
Operate. Give the candidate a trace and an incident timeline from a replayed fault, with personal data replaced by synthetic values. Ask what happened, what they would check next and what they would change in the runbook.
Govern. Give the candidate a one-paragraph AI usage policy and a description of a system. Ask which statements map to a control, which are untestable, and what evidence they would ask for.
Every sample uses synthetic or properly anonymised material. Never put real client data, real personal information or confidential code in front of a candidate.
Decide the AI-use rule before the interview
Candidates will ask whether they may use an AI assistant during the exercise. Decide the answer before the first interview, apply it to every candidate for that role, and tell candidates in the invitation.
There are three workable rules. Allowed and observed: the candidate may use an assistant, and the panel watches how they verify its output. Allowed with disclosure: the candidate may use an assistant and explains what it produced and what they changed. Not allowed for one part: a specific part of the exercise is done without assistance, because it tests reasoning the role needs unaided, such as explaining a trade-off.
CloudNala's view is that for most AI-era roles, allowed and observed is the most informative rule, because the job is to use AI and check it. Whatever rule you choose, the unfair version is the unstated one, where some candidates assume AI is forbidden and others assume it is expected.
Score against anchors, not impressions
An anchor is a written description of what a weak, sound and strong answer looks like for each criterion. Anchors make a score of 3 mean the same thing to every panellist.
Sabie's rules are simple. Each panellist scores alone, in writing, before the panel meets. The panel then compares scores criterion by criterion and discusses only where they differ by two points or more. The final decision records the scores and a sentence of reasoning per criterion.
This does three things. It reduces the influence of the most confident person in the room. It shows over time whether panellists are calibrated with each other. And it gives an unsuccessful candidate specific, honest feedback: "You found the failures but did not identify the pattern behind them" is far more useful than "we went with someone with more experience".
The work-products article makes a related point about promotion: the evidence used to hire someone should look like the evidence used to judge their work later.
Fairness and the law, in outline
Work samples and scored exercises are assessments, and in South Africa assessing applicants is regulated. The points below are a summary for a non-legal audience, checked against the legislation on 31 October 2026.
The Employment Equity Act applies to applicants. Section 9 states that for the purposes of sections 6, 7 and 8, "employee" includes an applicant for employment. Section 6 prohibits unfair discrimination, and section 8 deals with psychological testing and other similar assessments.
Section 8 sets three conditions. Psychological testing and other similar assessments of an employee are prohibited unless the test or assessment has been scientifically shown to be valid and reliable, can be applied fairly to all employees, and is not biased against any employee or group. A former paragraph (d), which required certification by the Health Professions Council of South Africa or another authorised body, was deleted by section 2 of the Employment Equity Amendment Act 4 of 2022, with effect from 1 January 2025 under the commencement proclamation. That deletion does not settle any separate professional rules about who may administer psychological tests, which this article does not cover. Whether a particular work sample counts as a "similar assessment" is a question for your legal and HR advisers. Designing every exercise as though the three conditions apply is a sensible default.
In practice that means keeping samples job-related (traced to the role profile), consistent (same task, same rule on AI use and same anchors for every candidate for the role), checked for bias (does any part depend on something unrelated to the job, such as a particular accent in a recorded call, or access to a fast home connection?) and accessible, with reasonable accommodation for candidates who need it.
POPIA applies to candidate data. CVs, interview notes, recordings and scores are personal information. In outline, the Act expects you to tell candidates what you collect and why, to keep it only as long as you need it for the recruitment purpose or as another law requires, and to secure it. Section 71 also restricts decisions with legal or substantial effect that are based solely on automated processing intended to profile a person, including their performance at work, subject to exceptions in the section. CloudNala's recommendation goes further than the minimum: if you use an AI tool to screen CVs or summarise interviews, tell candidates, keep a person making the decision, check what data the tool sends where, and do not paste CVs into unapproved services. The confidentiality article covers how to decide what may enter which AI service. The Act itself is published on the government's website.
Keep a record of the decision
For each hire, Sabie keeps a short decision record: the role profile used, the work samples and AI-use rule, each panellist's anchored scores and reasons, and the final decision. The record is stored with the recruitment file and kept under the same retention rules as other candidate information.
The record serves three purposes. It lets the organisation explain a decision if a candidate asks. It shows over time whether the work samples predict success in the role, which is the real test of whether they are valid. And it makes the process improvable, because a weak sample shows up as one that never separates candidates.
Start this week
- Take one open AI-related role and write its core verbs, decisions owned and work products, using Part 2's template.
- Design one work sample for the most important verb, using synthetic material, and write anchors for weak, sound and strong answers.
- Decide and write down the AI-use rule for that role, then ask HR and legal to review the exercise before it is used.
This is the final part. Start again with Part 1, AI roles decoded: read the verbs, not the title, or see all parts on the series page.
How CloudNala can help
CloudNala helps technology leaders design selection processes for AI-era roles: role profiles built on verbs and work products, realistic work samples on synthetic material, a clear rule on AI use, anchored scoring sheets and a decision record. We work alongside your HR and legal advisers rather than replacing them, and we do not provide recruitment or psychometric assessment services.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Capability Planning Session or write to us at consult@cloudnala.co.za