In the third week of the night shift at Keurbooms Software, the list had a new kind of task on it. Ayesha, the product owner, had heard from a large tour operator that Lagoonbook slowed down during the December booking rush. She added a line to the night list before leaving: "Improve scalability of the booking flow." Nobody questioned it. The flag removals had gone well and the agents seemed capable.
At 06:15 there was one pull request. It was large, around 4,800 changed lines, and it was impressive. The agent had taken Lagoonbook's four main modules, bookings, payments, availability and notifications, and extracted each into its own service with its own deployment configuration, a message queue between bookings and notifications, HTTP clients replacing in-process calls, and updated tests. The pipeline was green. The description explained, accurately, that each service could now scale independently.
Thandeka, the tech lead, read the description twice and then called Pieter, who carries the on-call phone. "If we merge this," she asked, "who gets woken up, and for what?" They spent the morning answering that question instead of reviewing the code. The pull request was closed before lunch. This episode is about why.
As in the rest of this series, Keurbooms Software is fictional: a nine-person team in Plettenberg Bay running a booking and payments platform for guesthouses and tour operators. The figures are illustrative.
A clean diff with an unpriced bill
The code was not the problem. The agent had done what it was asked, in the way many architecture articles suggest. The problem was everything the diff did not contain.
The card charge itself has always gone to an external payment provider, outside Lagoonbook's database. But before the change, a booking and its payment record were written in one database transaction inside one process: either both existed or neither did. After the change, an internal network hop sat between them, with its own timeouts and retries. A timeout between the bookings and payments services could now leave a booking confirmed with its payment record missing or unknown. The agent had added a retry, which raised a new question: if the first call had actually succeeded and only the response was lost, would the retry create a second payment record, or a second charge request to the provider? Our article on why a successful payment can still end in a failed order walks through exactly this class of failure.
The morning review listed the responsibilities the change would have created:
- Network behaviour. Timeouts, retries, idempotency keys and partial failures on every call that used to be a function call.
- Consistency. A booking and its payment record no longer committed together. The team would need a compensation process or an outbox pattern, and a way to find and repair mismatches.
- Observability. Four services meant four sets of logs, metrics and alerts, and a trace that had to follow a request across all of them. Our guide to logs, metrics and traces when production breaks shows how much of an incident is spent just reconstructing that path.
- Environments and pipelines. Four deployment pipelines, four sets of configuration and secrets, and test environments that now needed all four services running.
- Cost. More always-on compute, a managed queue and more data transfer. Small numbers individually, but a permanent increase for a nine-person company. As we have written before, serverless and smaller units are not automatically cheaper.
- Support. Pieter's on-call rota would cover four failure domains instead of one.
None of this appeared in the 4,800 lines because none of it is code. It is operating responsibility. The diff was the smallest part of the change.
Was the agent wrong?
Not exactly. "Improve scalability" is a goal, not a task. It does not say what is slow, under what load, or what improvement is enough. Given an open goal, the agent produced a common, well-documented answer. If you ask a capable engineer the same vague question late at night with no one to consult, you might get the same answer.
What made it the wrong answer was context the agent did not have. Pieter pulled up the December traces. The slowdown came from one place: availability lookups for multi-property tour bookings ran a query per property per night of the stay. One index and one batched query would remove most of it. That is a small, testable, reversible change, exactly the kind of work that suits the night shift, once someone has diagnosed the problem in daylight.
This is a pattern worth naming. The modular structure Lagoonbook already had was good. Our article on when a module should become a microservice sets out five tests to apply before any extraction: does the module change on its own rhythm; can one team own it fully; can it own its data without shared joins; does it need different scaling or availability; and are tracing and on-call ready for a network hop? The tests are cumulative, not alternatives: if any answer is no, the better move is usually to tidy the boundary inside the application. Lagoonbook failed most of them. All four modules changed together, the same nine people owned everything, payments and bookings shared tables, and Pieter had no distributed tracing. Only the scaling question was even arguable, and the traces answered it. The booking flow needed a better query, not a distributed system.
The rule: architecture changes need a decision record, not an overnight pull request
The team made one change to the night list after this morning. Every proposed task is now sorted before it is queued.
The test is simple: is the change small, reversible, owned by one person and checkable by a test or by telemetry? If yes, it can go on the night list. If it changes deployment units, data ownership, transaction boundaries or runtime behaviour across modules, it does not. It goes through a short architecture decision record first, written in daylight with the people who will run the result.
For Keurbooms, that record has five parts:
- The problem, with evidence. What is slow or failing, measured where, under what load.
- Options considered. Including the boring one. In this case: add an index, batch the query, cache availability, or extract a service.
- Operating consequences of each option. Who gets paged, what new failure modes exist, what it costs to run.
- The decision and its owner.
- How the change will be staged and how success will be measured.
The agent can help with the record. It is good at drafting options, listing failure modes and estimating the scope of a change. What it cannot do is own the decision, because the decision is about what the team is willing to operate. The architecture delta, a clear statement of what changes in the system and what stays the same, is the document the reviewers actually needed that morning. The night produced code without one.
Why vague goals are dangerous at night specifically
During the day, an engineer handed "improve scalability" would ask questions within minutes: slow where, for whom, how much better? At night there is nobody to answer. Background agents fill that gap with assumptions, and the more capable the agent, the more convincing the result.
Keurbooms now writes every night task in a fixed shape:
- Change: one concrete thing to do.
- Boundary: the files or module it may touch.
- Done when: the test, metric or telemetry that proves it.
- Not allowed: anything the agent must not change, stated explicitly (for this task class: no new deployment units, no changes to transaction boundaries).
If a task cannot be written in that shape, it is not ready for the night list. That single rule would have kept "improve scalability" off it. The same discipline appears in our article on specifications for AI agent workflows: an agent given a precise task behaves precisely, and one given a slogan behaves creatively.
What happened next
The following week, after Pieter's diagnosis, two tasks went on the night list: add a composite index to the availability table, and replace the per-night query with a batched one behind a flag. Both came back as small pull requests with benchmarks attached, recorded before and after against a copy of December's traffic. Both were merged. The booking flow's slowest step dropped from seconds to a few hundred milliseconds in the benchmark. That is a fictional number, but a realistic shape for this kind of fix.
The closed 4,800-line pull request was kept, not deleted. It is a useful exhibit. When someone on the team proposes splitting a module in future, the review starts by asking the questions that pull request could not answer.
Key points
- An architecture change can produce a clean, passing diff while hiding a large operating bill: network failure, consistency, observability, environments and cost.
- Vague goals such as "improve scalability" are not tasks. At night, agents fill the gaps with assumptions.
- Diagnose in daylight. The real fix here was an index and a batched query.
- Sort every task before it reaches the night list: small and reversible goes to agents; architecture goes to a decision record.
- Agents can draft the decision record; people own the decision, because it is about what they will operate.
Previous: Part 1, Let the agents take the night shift. Next: Part 3, The overnight upgrade that passed every test. All parts are on the series page.
How CloudNala can help
CloudNala helps teams decide when an architecture change is worth its operating cost. We review proposed extractions and refactors, whether drafted by people or by agents, against measured evidence, write the decision record with you, and set the task-intake rules that keep background agents on small, reversible work while architecture decisions stay with the people who will run the result.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Talk to us about coding agents in your delivery pipeline or write to us at consult@cloudnala.co.za