By its fifth week the night shift at Keurbooms Software had a reputation for being boring, which was the point. Flag removals, dead code, missing tests. Then Thandeka, the tech lead, added a category that seemed just as safe: dependency upgrades with no known breaking changes. On a Thursday night a background agent upgraded invoice-forge, the library Lagoonbook uses to produce guest invoices, from 2.3.1 to 2.4.0. It was a minor version. The agent read the library's release notes, found nothing alarming, ran the full suite and opened a pull request. All 1,412 unit tests passed. The contract tests passed. The evidence pack was complete.
Pieter, on call that week, reviewed it at 08:40 and was about to merge when he noticed the lock file diff was longer than he expected. invoice-forge 2.4.0 had raised its own dependency on a small money-handling library, ledger-kit, from 3.x to 4.0. Nobody at Keurbooms had ever chosen ledger-kit. It arrived as a dependency of a dependency.
He did not merge. By 11:00 the team had found that ledger-kit 4.0 changed its default rounding for half-cent amounts from half-up to half-even. For most bookings it made no difference. For a few, the deposit line on the guest invoice would now be one cent different from the deposit the guest had actually been charged.
As throughout this series, Keurbooms Software is a fictional Plettenberg Bay company running Lagoonbook, a booking and payments platform for guesthouses and tour operators. invoice-forge and ledger-kit are invented libraries, and all versions and counts are illustrative.
One cent is not a rounding error when it is money
Lagoonbook takes a deposit when a guest books, usually a percentage of the total. A 30% deposit on a R1,234.55 stay is R370.365. Under half-up rounding that becomes R370.37. Under half-even, sometimes called banker's rounding, it becomes R370.36, because the digit before the half is even.
Neither rule is wrong. Many financial systems use half-even deliberately because it avoids a small upward bias over many transactions. The problem is changing the rule silently, in one place only. Lagoonbook calculates the deposit itself, with half-up rounding, and asks the payment provider to charge that amount. The upgrade could not change what guests were charged. What it could change was the invoice: invoice-forge computes the invoice line items again, using ledger-kit, so after the upgrade a few invoices would show a deposit one cent different from the amount actually charged. The guest's invoice would no longer match their card statement, the guesthouse's reconciliation report would flag a mismatch, and a document meant to record the transaction accurately, possibly a tax invoice, would be wrong by R0.01. Multiply by a busy season and it is a support problem, a trust problem and an audit question.
Why every test passed
The tests at Keurbooms were good. They checked that invoices had the right line items, VAT at 15% and totals that matched the inputs. What they did not check was the rounding rule itself, because nobody had ever thought of it as a decision. It was simply how the library behaved. The test fixtures happened to use amounts that never landed exactly on a half-cent.
That is the general lesson of this episode. A passing test suite proves the behaviour the team specified. It says nothing about behaviour nobody specified. Our article on why a passed test is not proof of production readiness makes this point for releases in general. An upgrade is a particularly sharp case, because the change comes from someone else's decisions, sometimes two steps away.
The agent had done what it was asked: read the release notes of the library it was upgrading, run the tests and attach the evidence. The release notes for invoice-forge 2.4.0 did mention "updated ledger-kit to 4.0" in one line. They did not repeat ledger-kit's own note about the rounding default. The agent did not follow the chain, and its instructions did not ask it to.
How they caught it
Pieter's instinct about the lock file was the first signal. The second was a practice the team had adopted for the payments area after an unrelated incident: a shadow run.
A shadow run is an offline replay: a recorded sample of real requests goes through the old and new versions of the code side by side, and the outputs are compared. Keurbooms keeps a rolling, anonymised sample of bookings from the previous month: amounts, dates and rates, with guest details removed. The team generated invoices for all of them with the new version and compared each invoice deposit line against the deposit Lagoonbook had charged. Of 9,840 invoice deposits, 27 differed by R0.01, and every one was a half-cent case.
That comparison took about four minutes to run. It found something no test in the suite could have found, because it used real inputs rather than the inputs the developers had imagined. It is the same idea as keeping evidence from failed tests rather than discarding it, which we describe in why failed tests must keep their evidence: real data surfaces the behaviour you did not predict.
The sample is anonymised for a reason. Recorded booking traffic with guest names, phone numbers or card details in it is personal information, and a replay environment is rarely protected as carefully as production. Keurbooms strips identifying fields when the sample is captured and keeps only what the calculation needs. The agent's test environment never sees the original records.
The third check was reading the transitive release notes. Once the team knew ledger-kit had moved a major version, finding the rounding change took two minutes. The information was available. The process simply had not looked for it.
Dependencies are executable decisions someone else made
Our article on pinning the version before you trust the platform makes the case for knowing exactly which version you run. Transitive dependencies are where that knowledge usually runs out. Every dependency is code that runs with your application's permissions and makes decisions on your behalf. A transitive dependency is one you did not choose, did not review and may not know you have. Our series on the AI software supply chain makes this point about agent skills and MCP servers as executable dependencies and about the AI gateway as part of the supply chain. The same logic applies to an ordinary money library.
Background agents make upgrades cheap to propose. That is useful, because unpatched dependencies are a real risk. But cheap proposals make it easy to treat each upgrade as a formality. The agent can do the mechanical work very well: bump the version, update the lock file, run the tests, read the notes. What it needs from the team is a rule about which upgrades carry enough behavioural risk that mechanical checks are not enough.
The upgrade rule the night shift follows now
Keurbooms rewrote its upgrade rule the same afternoon.
- Patch upgrades of direct dependencies, with release notes read and no change to transitive dependencies: the night shift may open the pull request on its own.
- Minor upgrades, or any upgrade that changes a transitive dependency: the night shift may open the pull request, but it must include the full transitive change list from the lock file, the release notes for every changed package (not only the top one) and a shadow run on recorded traffic for any affected calculation. A named reviewer approves.
- Anything touching money, time zones, dates or authentication: daytime only, with a named owner and a shadow run, whatever the version number.
- Major versions and new dependencies: not night work. These need a short decision record.
The agent's instructions changed too. It now lists every package whose resolved version changed, summarises each package's release notes and flags any mention of defaults, rounding, locale, time zone or serialisation. That summary appears at the top of the evidence pack, above the test results.
The team also added the test that should have existed: a handful of bookings whose deposits land exactly on half-cents, checking that the invoice deposit line equals the charged deposit, with the expected rounding written down as a deliberate rule. Rounding is now a specified behaviour with an owner, Ayesha, because how Lagoonbook rounds guest money is a product decision, not a library detail. That is the habit our article on tests without a requirement argues for: a behaviour that matters should be traceable to someone's decision.
What the morning decided
The pull request was not merged as it stood. The team kept invoice-forge on 2.3.1 until the rounding decision was made. That afternoon's decision, with Ayesha as owner, was to keep half-up for both the charged deposit and the invoice, for consistency with what guesthouses already reconcile against, and to configure the rounding mode explicitly rather than inherit a library default. The upgrade then went ahead in daylight with that setting and the new half-cent tests.
No guest was ever charged a different amount, because the upgrade never touched the charge, and no invoice went out disagreeing with a charge. That was the outcome of a reviewer who read the lock file, a shadow run that used real inputs, and a rule that now makes both of those routine rather than lucky.
Key points
- A green pipeline proves the behaviour you specified, not all behaviour. Upgrades change behaviour you never specified.
- Transitive dependencies are decisions someone else made. Read their release notes too.
- Shadow runs on recorded, anonymised traffic catch differences that test fixtures never exercise.
- Let the night shift propose patch upgrades; require shadow runs and named approval when transitive dependencies change; keep money, time and authentication changes in daylight.
- When a hidden behaviour turns out to matter, specify it with a test and give it an owner.
Previous: Part 2, The night the agent split everything into services. Next: Part 4, What the night shift is allowed to merge. All parts are on the series page.
How CloudNala can help
CloudNala helps teams make dependency management safe enough to automate. We set up transitive change reporting, shadow runs on anonymised recorded traffic for critical calculations, and upgrade rules that tell background agents what they may propose alone and what needs a person, so that patches arrive quickly without behaviour changing quietly underneath you.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Talk to us about coding agents in your delivery pipeline or write to us at consult@cloudnala.co.za