The executive committee meeting lasted forty minutes and ended with one question. Bontebok Furniture Group, a fictional South African furniture retailer with stores and home delivery, had a successful pilot in one region, a modest but measured assistant, and a request to roll the whole thing out to all three distribution centres. The chief financial officer asked Riaan, the solution architect: "So what exactly are we building, and how will we know if we got it wrong?"
He could have answered with the pilot deck. It had good charts. But a deck records a moment, not a decision. Six months later, when a new vendor offered a "delivery AI platform" or a new executive asked why the chatbot did not handle refunds, the deck would not explain what had been decided, on what evidence, and what had been rejected and why.
Bontebok's people, systems and figures in this series are illustrative. This final part shows how its team turned discovery evidence into architecture decisions that could survive budget cycles, vendor pitches and changes of staff.
Evidence goes stale; decisions need to cite it
By the end of the pilot, the team held a lot of evidence: the demand analysis from part 1, the information gap audit from part 2, the mined process from part 3, the fit test from part 4 and the demand measure from part 5.
Evidence has a shelf life. Contact reasons shift when a supplier changes. A process mined in winter may not describe December. So the team kept each finding in an evidence register with its source, method, date, owner and confidence. Our article on discovery questions that change the answer describes an evidence register for bids; the same structure works here. The new step is what happens next: every architecture decision cites the findings it rests on, so that when a finding is re-measured, the decisions that depend on it are easy to find.
The chain runs in one direction: outcome, demand finding, root cause, decision, measure. If a decision cannot point back to a finding, it is a preference. If it cannot point forward to a measure, nobody will know whether it worked.
The decision record, extended
Michael Nygard's widely used format for architecture decision records (2011) has five parts: a short title, the context and forces at play, the decision in full sentences, a status, and the consequences, all of them, "not just the positive ones". Bontebok used that format and added five fields that discovery-led decisions need.
| Added field | Why it matters |
|---|---|
| Evidence | Links to the findings in the evidence register, with their dates |
| Options considered | Including doing nothing and at least one option without AI |
| Gate result | For any AI option, the outcome of the do-not-automate-yet gate from part 1 |
| Measure and owner | The outcome measure this decision should move, and who reports it |
| Reopen when | Specific conditions that would make the team revisit the decision |
The last field is the one that pays off later. A decision with explicit reopening conditions does not need to be defended against every new vendor pitch. The question becomes whether a condition has been met, not whether someone has a better idea.
Seven decisions from one discovery
The rollout produced seven decision records. Each fits on a page.
- A delivery status service, owned by logistics. An event-fed service holds the current state and expected window for every order. It is not a feature of the order system or of any chatbot platform, because several channels and teams depend on it. Evidence: the trapped and stale gaps from part 2.
- One authoritative source per fact. The warehouse system is authoritative until an order is scheduled, the route planner after that, and the carrier's events for carrier deliveries. Evidence: the contradictory gap from part 2.
- Proactive messages triggered by status events. Allocation, reminder, morning-of and date-change messages are sent through the existing messaging provider, triggered by the status service. Evidence: the "no news" and "date moved" demand from part 1.
- Exceptions in a workflow with named owners. Failed and partial deliveries open an exception immediately, assigned to a coordinator with a response time. Evidence: the conformance gap from part 3.
- A conversational assistant that consumes the status service. It answers replies to delivery messages, reads state but does not search notes or mailboxes, answers knowledge questions from owned content and takes two confirmed actions. Evidence: the fit test from part 4.
- A bounded model step for driver notes. It classifies free-text failure notes and proposes a next action; the coordinator decides. Evidence: the "Other" variant from part 3.
- No autonomous exceptions agent, for now. Deferred, not rejected. Evidence: the mined exception path is known and the delay was between steps, not inside them.
Two of the seven involve AI. Neither was the first thing built. Both depend on the first four.
One record in full
Decision 5 is the one most likely to be challenged by a future vendor, so here it is as it was written, lightly shortened.
Decision 5: The delivery assistant consumes the status service and does not own delivery data.
Status: Accepted for the pilot region; proposed for national rollout.
Context: Customers reply to delivery messages in free text in several languages. About 40% of remaining contacts are slot changes and about 25% are access instructions. Damage and complaints are about 20%. Delivery state is held in the status service (decision 1). Product and policy content has owners and review dates.
Options considered: (a) No assistant; staff handle replies. (b) Menu-driven replies with fixed buttons only. (c) Assistant over the status service and owned content, with two confirmed actions. (d) Vendor assistant with its own copy of order data and broader actions.
Decision: We will deploy option (c). The assistant reads state only from the status service for the customer linked to the order, answers knowledge questions only from owned content with a citation, and may rebook a slot or add access instructions after the customer confirms. All other requests are handed to a coordinator with the conversation attached.
Gate result: Passed all five conditions after the status service and exception workflow were live.
Consequences: Customers resolve slot changes without waiting. The assistant cannot be better than the status service. Option (d) would duplicate personal information and create a second source of delivery truth. Language testing must be repeated when the model or prompts change.
Measure and owner: Delivery failure demand per 100 orders due, reported with total delivery contacts per 100 orders due and the repeat-contact rate for assistant-handled requests; head of customer service.
Reopen when: Total delivery contacts in a region rise for two consecutive months after launch; a handed-over category exceeds 30% of assistant conversations; or the status service's accuracy falls below its agreed level.
Notice what the record does not contain: a model name, a vendor or a framework. Those belong in a separate, shorter-lived record that implements this one.
What the decisions changed outside IT
Several decisions reached beyond the technology team, which is usually a sign that discovery found the real problem.
Decision 2 became a procurement requirement. Carrier deliveries to outlying towns had been invisible since part 1 because the carrier's portal had no event feed. Bontebok's next carrier contract required delivery events to be sent to the status service within a stated time.
Decision 3 needed a privacy review. Messages about a customer's own order are not marketing, but the same messaging channel is often used for both. The Protection of Personal Information Act, 2013 (POPIA), section 69, restricts direct marketing by electronic communication, so Bontebok's information officer confirmed that delivery messages would carry no promotional content and that marketing would stay on its own consented path. That is a summary of Bontebok's internal decision, not legal advice.
Decision 4 changed a job description. Coordinators became owners of named exceptions with a response time, which needed a conversation with their manager and, in one distribution centre, an extra person.
Rolling out on evidence
The national rollout followed the decisions, one distribution centre at a time. Each new region kept a comparison period and the same demand measure from part 5. The status service and messages went first, the exception workflow second, the assistant only after the gate had been passed again for that region.
The evidence register kept growing. When Midrand's contact reasons turned out to differ (more complex access problems, more apartment deliveries), the team updated the findings and checked which decisions cited them. Decision 5 did not need to change. The reminder message did: it now asks for access details in the first line.
Our article on the architecture views needed to make a decision covers how to present options to an executive committee. The decision records are what remain after the presentation, and the architecture repository is where they live.
A closing checklist for the series
- The request is recorded, and the outcome behind it is written as something other than activity.
- Demand is counted across every channel and classified by cause, as failure or value demand.
- The information each answer needs is traced to its source, and its gaps are named.
- The process is measured from event data as well as observed.
- Every AI option has passed a written do-not-automate-yet gate.
- The smallest test ran against a comparison group.
- Every decision cites its evidence, names rejected options, states its measure and owner, and says when it should be reopened.
Key points
- A pilot deck records a moment. Decision records carry the evidence forward.
- Keep findings in an evidence register with dates and owners, and make every decision cite the findings it depends on.
- Extend the decision record with evidence, options including non-AI ones, gate result, measure and reopening conditions.
- Keep model and vendor choices in shorter-lived records that implement the architectural decision.
- Expect discovery-led decisions to change contracts, roles and privacy practice, not only systems.
This is the final part of Diagnose Before You Automate. For the outcome-first discipline in a banking setting, see Define the outcome before you build the assistant.
How CloudNala can help
CloudNala helps teams turn discovery into decisions that last. We set up an evidence register from your discovery work, facilitate the decisions with the people who own the outcomes, write decision records that name the evidence, the rejected options, the AI gate result, the measure and the reopening conditions, and plan a rollout that keeps measuring, so that your architecture can be explained to the next executive, auditor or vendor without reconstructing the argument.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Start a Digital Discovery Sprint or write to us at consult@cloudnala.co.za