Many organisations run data quality as a periodic project. A dashboard looks wrong, a consultant is brought in, records are cleaned, a report is written, and the problem returns within months because nothing about the daily operation of the data changed.
Software teams learnt long ago that reliability does not come from occasional clean-ups. It comes from operations: monitoring, alerts, on-call ownership, incident handling and learning from failures. Data that people and AI systems depend on needs the same treatment. CloudNala's view is that data quality is a property of how data is operated, not a state it is cleaned into, and that the most useful investment is usually not a new tool but moving existing checks to the right place and routing their alerts to the right people.
Gariep Grain Co-operative is fictional, and its systems and figures are illustrative. By now Gariep had a deliveries data product with an owner, a contract and checks (Part 1), an agreed definition of "delivered" (Part 2), and a member assistant and traceability tool built on top. The question for this part is what happens on the day something goes wrong.
Five kinds of check, in the right place
Most data quality failures fall into five kinds, and a small number of checks catches most of them:
- Schema: the expected columns are present with the expected types. Catches a source system that renamed a field or changed a unit.
- Volume: the number of rows is within the expected range for that source and day. Catches a partial export, a failed extract or a duplicated load.
- Freshness: the data arrived by the promised time. Catches a stalled pipeline before consumers start work.
- Referential integrity: records point to things that exist. Catches a ticket for a member who is not in the registry, or a load allocated to a bin that does not exist.
- Distribution: values fall within plausible ranges and patterns. Catches a moisture reading of 140 percent, or an average that has quietly shifted.
The kinds of check matter less than where they run. A check that runs after data has been published only tells you that consumers already have bad data. Checks should sit as early as they can still be meaningful.
At Gariep, the original checks all ran on the published table, overnight, and produced a report that the data team read in the morning. During the 2026 harvest, a power cut at one silo interrupted the weighbridge export, which arrived at a third of its normal size. The overnight check flagged it, but only after the payments run had already used the incomplete data to calculate advances. Thirty-odd members were underpaid and had to be corrected by hand.
After that incident the team moved the schema and volume checks to landing, where data first arrives, and made them blocking: a failing load is held, not published. The same power cut a month later stopped at landing. Payments ran on the previous complete day's data with a note, and the missing loads were added once the export was rerun.
Observability: knowing what the data is doing
Checks catch specific, known failure modes. Observability is broader: the ability to see how data is behaving over time so that unknown problems become visible. For data, that usually means tracking a small number of signals for each important dataset:
- arrival time against the service level;
- row counts by source over time;
- the share of records failing each check;
- distributions of key fields, such as moisture and mass by silo;
- schema changes, as events; and
- lineage, so any signal can be traced to its upstream cause and downstream impact.
None of this requires a specialised platform to begin with. Gariep's first version was a small table of these measures, appended on every load, and a dashboard the steward checked each morning. Dedicated data observability tools exist and can add automated anomaly detection, but the discipline matters more than the product. An expensive tool whose alerts nobody owns is worse than a simple table someone reads.
The same principle applies to AI systems built on the data. Observability for production AI tracks the model's behaviour; data observability tracks what the model is being fed. When an assistant's answers degrade, the cause is often upstream in the data, and you can only find it quickly if both are visible.
Incidents, not surprises
The most valuable change is to handle a data failure as an incident, with the same shape as a system outage.
Detect through a check or a signal, not through a consumer complaint.
Route the alert to the owner on duty. For a data product, that is the producing team, with the data owner informed for anything affecting consumers. It is not the platform team, unless the platform itself failed. Routing every data alert to a central team is how alerts end up ignored.
Assess impact through lineage. Which consumers use this data, and have they already read the bad version? At Gariep, lineage showed that the payments run, the member assistant and the BI dashboard all depended on the deliveries product.
Contain and communicate. Hold publication, and tell affected consumers before they notice. The member assistant was set to report "today's figures are being updated" rather than give a partial total.
Fix and backfill. Repair the source or the pipeline, and reload the affected period without duplicating data.
Review. Ask why the failure was not caught earlier, and add or adjust a check. This is the step most teams skip, and the one that makes next season better than this one.
Service levels make quality measurable
"Good quality" is not a target anyone can manage. A service level is. For each data product, Gariep's owner agreed a small set of commitments: complete for the previous day by 06:00 in harvest; fewer than one in a thousand tickets failing referential checks; no unannounced schema changes. These were measured from the observability table every day and reviewed monthly.
Service levels change conversations. Instead of arguing about whether the data is "bad", the owner and consumers can see whether commitments were met, how often, and what caused the misses. They also make it possible to decide where to invest: if freshness misses are almost all caused by one silo's connectivity, that is a clear, fundable fix.
The trade-off
Blocking checks protect consumers and occasionally delay them. Holding a partial load means the payments run uses yesterday's data. That is a business decision, and it belongs to the owner, not the data team. At Gariep, the owner decided that a delayed but correct advance was better than an on-time wrong one. In another context, publishing with a warning might be the right call.
There is also the cost of alert fatigue. Too many checks with sensitive thresholds produce alerts that people learn to ignore. Start with a few checks on the datasets that matter most, tune thresholds after the first few weeks, and treat a check that fires constantly without real problems as a defect to fix.
Closing the series
The six parts of this series make one argument from different angles. Data that people and AI systems can trust is not a matter of better tools or cleaner records. It is a matter of treating important datasets as products with owners, agreeing what they mean, choosing the right technique for each question, building skills through real work, and operating data with the same discipline as any production system.
If you take one action from the series, make it this: pick the dataset your most important AI or reporting use case depends on, give it an owner, and move its checks to where they can stop a bad load. Everything else in the series builds on those two changes.
This is the last part. Start again from Part 1, A data product is more than a governed table, or see all parts on the series page.
How CloudNala can help
We help organisations set up the operating side of data quality for the datasets that matter most: checks placed where they can stop bad loads, alerts routed to owners, lineage-based impact assessment and service levels the business can see. Typically this starts with one data product and a short incident review of a recent failure, so the improvements are grounded in what actually goes wrong.
Work with CloudNala
CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.
Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.
Book a Data Readiness Review or write to us at consult@cloudnala.co.za