Two pricing facts sit outside the model choice and the prompt design, and both are routinely ignored. The first is that AI providers sell the same computation at very different prices depending on how quickly you need the answer. The second is that, for a South African organisation, the price of that computation in rand changes every day.
Neither is complicated. Both are worth real money once volumes grow, and both are easier to design for at the start than to retrofit.
Tugela Claims Services is fictional, and its systems and figures are illustrative. Tugela's AI workloads now include the claims triage agent from earlier parts, plus several jobs that grew around it. Its combined AI invoice is about $6,000 a month.
The same computation, two prices
On 27 October 2026, two major model providers each published a batch interface at a 50% discount on both input and output tokens. In exchange, you submit requests as a group and collect results later. One provider stated that most batches finish within an hour and that results are available when all requests complete or after 24 hours, whichever comes first. The other stated a completion window of 24 hours, "and often more quickly". One also stated that its caching multipliers stack with the batch discount, and recommended its longer cache duration for batched work because batches can take longer than five minutes.
Those terms will change, so check them before relying on them. The principle will not: providers charge less when you let them schedule the work.
The trouble is that very few teams ask whether their work can wait. The prototype called the model synchronously, because that is the simplest code to write, and production inherited the pattern.
Ask how long the answer can wait
The decision is about latency tolerance, and it is a business question rather than a technical one.
When Tugela listed its AI workloads against that question, the picture changed.
| Workload | Who waits? | Can it wait hours? | Right mode |
|---|---|---|---|
| Triage when an assessor opens a claim | The assessor | No | Realtime |
| Summary of newly uploaded documents | Nobody immediately | Up to an hour | Async queue at the realtime price |
| Weekly re-score of all open claims against updated fraud indicators | Nobody | Yes, by Monday morning | Batch |
| Month-end quality sample of 1,000 closed claims | The quality team, next week | Yes | Batch |
| Evaluation runs for model and prompt changes | Engineers, the next day | Usually | Batch |
The weekly re-score was the surprise. It had been built as a loop of synchronous calls on a Sunday night, about 6,000 open claims a week. Nobody read the output until Monday. Moving it to batch halved its cost with no change to anyone's working day.
The evaluation runs are worth highlighting because every serious AI team has them, and they grow. Part 3 recommends re-running a 400-case evaluation on every model, prompt or router change. Three candidate models on 400 cases is 1,200 runs, several times a month. That is exactly the kind of work batch pricing is designed for.
What batch does not suit
Batch is not free money. It has three real costs.
Unpredictable completion time. "Most within an hour" is not a service level. If a downstream process depends on results by a fixed time, design for the stated maximum, not the typical case.
Failure handling moves. In a synchronous call, a failure is visible immediately to the code that made it. In a batch, individual requests can fail or expire while others succeed. Each result must be checked and failures resubmitted or escalated. Requests that expire unprocessed were not billed by the provider that documented this, but they still need handling.
Tool-using agents fit awkwardly. A batch is a set of independent requests. A multi-step agent that needs a tool result before its next call cannot run its loop inside one batch. Batch suits single-step classification, extraction, summarisation and scoring, or the first step of a longer process.
Between realtime and batch there is a middle option that costs nothing extra: an asynchronous queue that processes work at the normal price but off the user's critical path. It does not save money per call, but it smooths load, avoids rate-limit spikes and makes moving to batch later a configuration change rather than a rewrite.
The rand is a variable in your run cost
AI services are almost always priced in US dollars. Budgets in South Africa are approved in rand. The gap between the two is a live variable, and it moves without anyone touching the system.
On Tugela's $6,000 monthly bill, a rand moving between R16.50 and R19.50 to the dollar changes the monthly cost by R18,000, about 17% of the bill at R18, with no change in usage. Over a financial year that is a range of more than R200,000 on a modest AI estate. For a large one, it is a treasury conversation.
This is not a reason to avoid dollar-priced services. It is a reason to budget for them properly:
- Forecast in a range. State the exchange-rate assumption explicitly and show the cost at a weaker and a stronger rand, as Part 6 does for pilot quotes.
- Set alerts in rand. Finance will ask about rand. A dollar threshold that never trips while the rand weakens is not a control.
- Separate usage variance from currency variance in monthly reporting. "Cost rose 12%" should be broken into how much was more work and how much was the exchange rate, otherwise the engineering team gets asked to explain something it did not cause.
- Read the billing terms. Some services are invoiced locally in rand through a reseller or cloud marketplace. That changes who carries short-term currency risk and when prices are reset, but rarely removes the link to the dollar price. Ask how often the rand price is reviewed.
- Treat commitments carefully. Prepaid or committed spend can lock a price per token, but it is usually denominated in dollars, so it fixes the usage price and leaves the currency exposure in place. Renting a lane vs paying per trip covers when commitments make sense at all.
Residency can carry its own premium
Where data must be processed in a particular region, the location of inference may also change the price. On 27 October 2026, one provider published a 10% premium for regional rather than global endpoints on the partner clouds that offer its models, and a 1.1 times multiplier on all token prices when inference is restricted to the United States on its own platform.
Two separate questions are tangled up here. The first is whether your data actually needs to stay in a given region, which is often less strict than assumed and is covered in POPIA does not say your data must stay in South Africa. The second is which models are actually offered in that region at all, which is covered in AI models available in South Africa. Only once both are answered does the premium become a straightforward line in the business case. Keeping AI in the country explains the wider cost of sovereignty choices.
Putting it together for Tugela
After reclassifying its workloads, Tugela moved the weekly re-score, the month-end quality sample and all evaluation runs to batch. Together they were a little over a third of the invoice, so the move reduced the monthly bill by about 17%. Triage stayed realtime, because an assessor is waiting.
The monthly AI report now shows cost in rand at the actual average exchange rate, alongside the cost at the budget rate. The difference is labelled "currency", and it is no longer anyone's performance issue.
Next: Part 6, Quote the AI run cost before the pilot starts. All parts are on the series page.
How CloudNala can help
We review AI workloads against latency tolerance, identify the jobs that can move to batch or asynchronous processing, and set up reporting that separates usage variance from currency variance and states the residency premium explicitly. The changes are usually small in code and noticeable on the invoice.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book an AI Cost Review or write to us at consult@cloudnala.co.za