Digital Transformation5 November 20269 min read

Diagnose Before You Automate — Part 3 of 6

Process mining before agent design

Before an agent is designed for a process, the process should be measured from its own event data. Part 3 of Diagnose Before You Automate shows how to build an event log from operational systems, what discovery and conformance reveal, and how each mined variant points to a rule, a workflow, an upstream fix or, occasionally, an agent.

#Digital Transformation#Process Design#AI Agents#Architecture Discovery#AI Adoption
A large lens on a stand over a network of delivery routes reveals a single amber-marked break in one route.

Once Bontebok Furniture Group's delivery status service was running in its pilot region, the logistics director had a new idea. Bontebok is a fictional South African furniture retailer with stores and home delivery, and everything about it in this series is illustrative. His idea was an exceptions agent: software that would pick up every failed or rescheduled delivery, work out what had gone wrong, contact the customer, find a new slot and rebook the truck, without a coordinator in the loop.

It was a more serious proposal than the chatbot in part 1. Exceptions were where customers suffered most and where coordinators spent their days. But the proposal described the process the way the logistics director believed it ran. Nobody had yet looked at how it actually ran, case by case, from the systems' own records.

That is what process mining is for. This part shows how Bontebok built an event log from its delivery systems, what it learned, and how each finding changed the agent proposal.

What process mining is, briefly

Process mining analyses the records that systems already keep about work: which activity happened, to which case, and when. The Process Mining Manifesto, published by the IEEE Task Force on Process Mining in 2011, describes three types of process mining:

  • Discovery builds a model of the process from the event log alone, without assuming what the process should look like.
  • Conformance checking compares an existing model, such as the documented procedure, with the event log, and shows where reality departs from it.
  • Enhancement extends an existing model with information from the log, such as where time is lost.

The interchange format most process-mining tools understand is XES, first standardised as IEEE 1849-2016; its first revision, IEEE 1849-2023, was published in August 2023. Tools and formats are secondary here. The discipline is what matters: let the records show the process before anyone designs automation for it.

This complements observation rather than replacing it. Our article on why the process map is usually fiction explains how to observe people doing the work. Process mining adds volume and timing: how often each path occurs and where the waiting happens, across every case rather than the twenty you can shadow.

Building the event log

Fatima, Bontebok's data analyst, and Riaan, its solution architect, spent most of the effort on the event log rather than the analysis. That is normal.

From operational systems to an event logSOURCESPoint of saleWarehouse systemRoute plannerDriver appException workflowContact channelsEVENT LOGCase: order (main)Linked: items, deliveriesActivityTimestamp: device timePersonal data removedANALYSISDiscovery: thepaths that occurConformance: wherereality departsEnhancement: wheretime is lost
Six systems each record part of a delivery. Joined into one event log, with a chosen case, an activity and a trustworthy timestamp for every event, they support discovery, conformance checking and enhancement. Most of the effort sits in the log itself: choosing the case, linking orders, items and deliveries, and using device time rather than upload time.

They took twelve weeks of events from the pilot region, covering about 9,700 orders: the six weeks before the pilot in part 1 and the six weeks of it. The events came from six sources:

SourceActivities recorded
Point-of-sale systemOrder placed, delivery date promised
Warehouse management systemStock allocated, short-picked, picked, loaded
Route plannerDelivery scheduled, rescheduled
Driver appOut for delivery, delivered, delivery failed (with reason and note)
Exception workflowException opened, customer contacted, rebooked, closed
Contact centre and WhatsAppCustomer contact, by classified reason

Three problems took most of the time, and each is common.

Choosing the case. A traditional event log assumes every event belongs to one case. Bontebok's work has several kinds of object at once: an order, the items on it, and the deliveries that carry them. A three-item order can arrive in two deliveries, and one truck carries many orders. Treating the order as the case makes a split delivery look like a strange loop. Treating the delivery as the case loses the customer's view. The team kept both, linked, which is what object-centric event log formats such as OCEL 2.0 are designed to represent. For a first analysis they took the order as the main case and flagged splits explicitly.

Recorded time is not event time. Drivers in parts of the Eastern Cape lose signal on some routes. The driver app queues events and uploads them when it reconnects, stamped with the upload time. Until the team switched to the device's own timestamp, some deliveries appeared to happen after the customer had already phoned to say the truck had come.

Free text is evidence too. The driver app's failure reason was a short pick list, often with a typed note. "Other" was the most common pick, and the note usually explained what had happened. Those notes became important later.

Personal information was kept out of the log. Orders were identified by a pseudonymous key, addresses were reduced to a suburb and distribution centre, and the notes were reviewed before analysis.

What discovery showed

Discovery produced a process model with far more paths than the logistics director's description. Illustratively:

  • 61% of orders followed the main path: placed, allocated, picked, scheduled, loaded, delivered, once each.
  • 17% were rescheduled at least once, and 6% two or more times. Most reschedules were caused by stock arriving late at the distribution centre, not by anything in the delivery operation.
  • 11% were split into more than one delivery because an item was short-picked.
  • 5% had at least one failed delivery attempt.

The remaining orders were a long tail of small variants: cancellations, address changes, collections from store.

The overlay of customer contacts was the most useful view. Contacts clustered after reschedules, especially second reschedules, and in the days after a failed delivery. Orders on the main path generated very few contacts once proactive messages were in place.

What conformance showed

Bontebok's documented procedure said a failed delivery would be rebooked within two working days. Conformance checking against the log told a different story. Before the pilot, the median time from a failed attempt to a rebooked slot was four and a half working days, and about a quarter of failed deliveries waited more than a week. During the pilot the median improved to about three working days, still outside the procedure.

Enhancement located the delay. Almost none of it was in the rebooking itself. The pilot's rule that a coordinator contacts the customer within one working day was being met, but its clock started only when the coordinator received the failure, and failures still reached coordinators in the depot's daily batch email for most of the pilot. The rest of the delay sat between the coordinator's first attempt to call the customer and actually reaching them.

That finding alone changed the agent proposal. An agent placed after the coordinator's queue would have inherited a delay it could not see.

From mined variants to design responses

Unlike the variant table in the process map is usually fiction, which came from an observed sample, this one comes from mined volumes and timings across every order in the window. Riaan's team took each significant variant and asked the question from our article on using a workflow until the unknown forces an agent: is the uncertainty in a step, in the path, or nowhere material?

Mined variants map to design responsesMINED VARIANTDESIGN RESPONSERescheduled: stock lateFailed: damaged at loadingSplit after short-pickFailed: access problemsFailed: customer not homeDelay before coordinatorFailed: reason "Other"Fix upstreamRule at the tillor at allocationWorkflow: reminderwith confirm or rebookIntegration: failureopens exception nowBounded model step:classify the note
Each significant variant from the mined delivery process gets its own design response. Most are upstream fixes, rules, workflows or integrations. Only the free-text failure notes, where wording genuinely varies, justify a model, and that step classifies and proposes while a coordinator decides.
VariantFindingDesign response
Rescheduled because stock arrived lateCause is upstream, in supplier lead timesFix upstream: warn at the till when stock is not yet in the distribution centre; send a date-change message automatically
Split delivery after short-pickPredictable from warehouse stock levelsRule: hold the order or ask the customer at allocation whether to split
Failed: customer not homeAbout half of failures; before the pilot there was no reminder, and the pilot's reminder had no way to confirm or rebookWorkflow: reminder with a confirm or rebook link the day before
Failed: access problemsGate codes, complexes, no liftRule and data: capture access details at the till and on the reminder
Failed: damaged at loadingSmall but costlyFix upstream: warehouse loading checks
Failed: reason recorded as "Other"Free-text notes vary widelyBounded agent step: classify the note and propose a next action for the coordinator
Delay between failure and coordinatorBatch email from the depotIntegration: driver failure creates the exception immediately

Only one row needed a model. The driver notes were genuinely varied ("complex security would not let us in, no answer on intercom", "customer says bought wrong size, wants exchange"), and a model could classify them and suggest the next step faster than a coordinator could read them. That step got a narrow contract: one question, read-only access to the order and note, a typed answer, and no authority to contact the customer or rebook anything. The coordinator decides.

Everything else was a rule, a workflow, an integration or a fix in somebody else's part of the business.

What happened to the exceptions agent

The proposal for an autonomous exceptions agent was not rejected outright. It was reduced to what the evidence supported. The path through an exception turned out to be well known; what varied was the reason at the start and the waiting in the middle. Removing the batch email and adding a confirm or rebook link to the reminder cut the volume of failed deliveries before any model was involved. Several variants also pointed to steps to remove or merge rather than automate, the treatment order set out in digitising bureaucracy is not transformation. The bounded classification step then helped coordinators work through the rest.

There was a second reason for caution. An agent that rebooks deliveries over several days has to hold state across calls, retries and customer replies. Our article on why an agent without durable state cannot own a long-running process explains why that belongs in a workflow engine, with the model called at specific points.

Limits of the method

Process mining only sees what systems record. At Bontebok, drivers and coordinators sometimes agreed new slots on WhatsApp and updated the route planner later, or not at all. Those cases looked like unexplained reschedules in the log. Observation and a few conversations with drivers explained them.

Data quality matters as much as it does anywhere else. Our article on data quality and observability as operational disciplines covers the checks; for an event log the critical ones are missing events, duplicated events and timestamps from different clocks.

And a mined model describes the past. When Bontebok changed the process, the model had to be mined again to see whether the new design was followed.

A checklist before agent design

  • Is there an event log for the process, with a clear case, activity and timestamp for every event?
  • Have you decided how to handle several kinds of object, such as orders, items and deliveries?
  • Do recorded timestamps reflect when events happened, not when they were uploaded?
  • Has discovery shown the main path and the variants that matter, with volumes?
  • Has conformance checking compared the documented procedure with what happened?
  • Have you located where time is lost, and whether it is inside a step or between steps?
  • For each significant variant, have you chosen between a rule, a workflow, an integration, an upstream fix and a model?
  • Is personal information excluded or pseudonymised in the log?

Key points

  • Measure the process from its own event data before designing an agent for it.
  • Expect most of the effort to go into the event log: choosing the case, handling several object types, and trusting timestamps.
  • Discovery shows the variants; conformance shows where reality departs from the procedure; enhancement shows where time is lost.
  • Map each significant variant to a design response. Most will be rules, workflows, integrations or upstream fixes.
  • Give a model only the steps where the uncertainty is real, with a narrow contract and no authority to act.

Next: Part 4, When a chatbot is the right answer. All parts are on the series page.

How CloudNala can help

CloudNala helps teams mine a process before they automate it. We build a working event log from your operational systems, run discovery and conformance analysis with your process owners, locate where time and demand are created, and map each variant to a rule, workflow, integration or bounded AI step, so that any agent you build is designed around how the work actually runs.


Work with CloudNala

CloudNala helps organisations move from technology ambition to practical execution across cloud, AI, data, platform engineering and digital services.

Whether you are exploring AI, modernising your cloud environment, building a public-sector digital service, or turning an idea into a working MVP, we can help you shape the roadmap and deliver the next step.

Start a Digital Discovery Sprint or write to us at consult@cloudnala.co.za