The first organisational response to AI tools was remarkably consistent. Buy licences for a coding assistant or a general chat tool. Run a half-day workshop on writing better prompts. Publish an acceptable-use policy. Report the number of people trained and the percentage of licences in weekly use.
By those measures most programmes look successful. Attendance is high, usage dashboards are green, and people say the tools help. Then someone asks the question a board or an executive committee eventually asks: what has changed in how we deliver? Are changes reaching production faster? Are there fewer defects? Is the work any safer? The honest answer is usually that nobody knows, and where somebody has measured, the change is smaller than the usage numbers suggested.
This series is about the gap between those two pictures. It is written for heads of engineering, CIOs, architecture and testing leads, and the people who run learning and development in technology functions. Its argument is simple to state:
AI capability belongs to the delivery system, not to individuals. Training should follow the work, and capability should be measured by what the work produces.
A companion series, Build Your Way into AI Engineering, covers the individual path: how one developer or architect builds credible skill by growing one system. This series covers the organisational path: what changes when four hundred people, in seven different roles, all start working alongside agents at once.
The running example
Kalahari Assurance is fictional, and its systems and figures are illustrative. Kalahari is the group IT function of a mid-sized South African insurer, based in Rosebank, Johannesburg. It has about 400 people: 190 engineers, 70 testers, 60 business analysts, 25 architects, 20 in the platform team, 15 designers and 20 managers and leads. It runs the policy administration system, the claims platform, a customer portal, a broker portal and a large set of integrations with reinsurers, banks and medical service providers.
In 2025 Kalahari did what most organisations did. It bought coding assistant licences for 300 people, ran a prompting workshop for 350 and wrote a one-page acceptable-use policy. Twelve months later the usage dashboard showed about 80 per cent of licences active in a typical week.
Three things also happened that the dashboard did not show:
- A business analyst pasted a section of policy wording, with a real policyholder's details in the example, into a personal chat tool to get a cleaner summary.
- A developer built a small MCP server that let an assistant read claims documents. It worked well. The developer then left, and nobody knew who owned the server, what it could reach or which applications used it.
- An agent-generated test suite for a claims rule passed in full while missing the rule that mattered most: a claim must not be closed while a fraud referral is open.
None of these is dramatic. Each one is what happens when individuals get capable tools and the system around them does not change.
Why the workshop model plateaus
A prompting workshop teaches a useful skill. It is also aimed at the wrong unit.
It treats AI as a personal productivity tool, so it trains the person holding the tool. But most of the work in a technology function is not done by one person. A change moves from intent, through specification, build, verification and release, into operation. Each stage is owned by a different role and each handover loses some information. When an agent starts producing material at one stage, every other stage is affected: reviewers see more code, testers see more change, analysts see requirements drafted by a model and operations see runbooks nobody wrote by hand.
The workshop also teaches one tool at one moment. Tools change every few months. A technique for one assistant's chat window is out of date by the time the next version ships, and the organisation has no mechanism to notice that the material it taught is now wrong.
Finally, the workshop produces the wrong evidence. Attendance and usage are easy to count, and they say almost nothing about whether work is better. A team can use an assistant every day and still ship the same defects more quickly.
What capability actually means
CloudNala uses a narrower definition than most training catalogues:
Organisational AI engineering capability is the ability of the delivery system to use agents and models to produce work that is correct, safe, owned and explainable, repeatedly, across teams, after the people who set it up have moved on.
Each word in that definition rules something out.
Delivery system rules out a programme that only trains developers. Correct, safe, owned and explainable rules out measuring speed alone. Repeatedly, across teams rules out one excellent team carrying the organisation's reputation. After the people who set it up have moved on rules out capability that lives in one enthusiast's head or one undocumented MCP server.
Shared foundations, different depth
The practical consequence is a curriculum with two layers.
The bottom layer is shared by every role. Kalahari settled on seven foundations:
- Intent. What outcome a change is for, who it serves and how success is measured.
- Specifications. How intent becomes acceptance criteria, constraints and boundaries precise enough for an agent to work from.
- Tool boundaries. What an agent may read, change, call and spend, and who set those limits.
- Evaluation. How you know an output is right, including the cases where it is wrong.
- Security. Prompt injection, data classes, secrets and least privilege, in plain terms.
- Cost. What a model call, a session and a completed task cost, and why that matters.
- Accountable decisions. Which decisions an agent may prepare and which a named person must make.
Every role learns all seven. An analyst who has never thought about tool boundaries will write specifications that leave them undefined. A tester who does not understand cost will design an evaluation suite nobody can afford to run on every change. A leader who does not understand evaluation will approve a pilot on the strength of a demonstration.
The top layer adds depth where each role's work needs it. Engineers go deeper into harnesses, tests and tool design. Architects go deeper into boundaries, gateways and decision records. Analysts into specification and acceptance. Testers into evaluation design and evidence. Designers into the human side of approval and correction. The platform team into paved roads, registries and defaults. Leaders into risk ownership and portfolio decisions.
The shared layer is what makes the rest work. When an analyst, a tester and an engineer share a vocabulary for intent, criteria and boundaries, they can review an agent's output together instead of each checking a different thing.
Training should follow the work
The second design rule is that learning should be attached to real work rather than delivered in a separate classroom.
At Kalahari this means three things. First, each module is anchored to an artefact the learner already produces: a specification, a test plan, a pull request, a runbook, a decision record. Second, the exercises use Kalahari's own systems in a safe environment, so the examples carry the organisation's real constraints. Third, the module is only complete when the learner has produced a reviewed piece of work using what was taught.
This is slower to design than a workshop. It is also the only way the learning survives contact with Monday morning.
Capability needs owners and standards
Training alone does not create capability, because what people learn has to land somewhere. If the organisation has no shared standards, every team invents its own conventions for repository instructions, agent permissions and evaluation, and the next team cannot reuse any of it.
So the programme has three owners, not one:
| Owner | Responsible for | Produces |
|---|---|---|
| Learning lead with the head of engineering | Curriculum, role paths, capstones | Modules tied to work products |
| Platform team | Toolchain, registries, defaults | A governed paved road that makes the standard easy |
| AI guild | Standards and reference implementations | Versioned standards with review dates |
Later parts take each of these in turn. Part 4 covers the move from personal tools to a governed toolchain. Part 5 looks at one specific owned asset, the internal MCP registry. Part 6 is about making the guild produce standards rather than enthusiasm.
How to tell whether it is working
If attendance and usage are the wrong measures, what are the right ones? Part 7 covers this in detail, but the principle is that capability shows up in work products. Kalahari tracks a small set of observable signals:
- the share of changes whose specification includes acceptance criteria an agent can test against;
- the share of agent-produced changes that pass local checks before human review;
- review time per accepted change, which should fall once harnesses mature;
- escaped defects traced to misunderstood intent;
- the number of AI assets (servers, skills, prompts, evaluation sets) with a named owner and a review date.
None of these is a training metric. All of them change when training works.
Where to start
Kalahari did not cancel its licences or its workshop. It reframed them. The workshop became the first hour of the shared foundations, not the whole programme. The licence dashboard stayed, but it stopped being reported as a success measure on its own. And the three incidents above became the first three teaching cases, anonymised inside the organisation, because they showed people exactly why the foundations matter.
If you are where Kalahari was in 2025, three questions are a reasonable first test:
- Could your testers, analysts and architects explain, in their own terms, what an agent is allowed to do in your delivery pipeline?
- Does anyone own the AI assets your teams have built, and could you list them?
- If you stopped reporting usage, what evidence would remain that your AI investment improved delivery?
If those questions are hard to answer, the gap is not a lack of enthusiasm or tools. It is the absence of organisational capability, and that is what the rest of this series is about.
Next: Part 2, Train the delivery system, not only the developer. All parts are on the series page.
How CloudNala can help
We help technology leaders design AI engineering capability programmes around their own delivery system: the shared foundations, the role paths, the work products each module should produce and the standards and platform defaults that make the learning stick. We start from the work you already do and the evidence you already have, and we are explicit about what a programme can and cannot change in the first year.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Plan an AI Capability Programme or write to us at consult@cloudnala.co.za