An automation that crashes with an error message is essentially good news. You know about it within a minute, someone fixes it, and the process carries on. The real problem starts when nothing visible happens at all — the scenario stops running, or keeps running but produces the wrong output, and for three weeks nobody notices. Invoices are not generated, orders are not matched, data between systems drifts apart, notifications go nowhere. When it finally surfaces, the fix is no longer a two-hour technical task but an operational exercise affecting customers, accounting and sometimes compliance.
This is why automation monitoring is not an add-on to be sorted out "some time after go-live". It is part of designing the process, just like the logic itself. An automated process without monitoring is simply a faster way to make mistakes at scale — and the more reliably it runs for the first few months, the less anyone checks it, and the longer it takes before you notice a failure.
Why a silent failure costs more than a crash
A loud failure has one enormous advantage: it is visible. The integration returns an error, the scenario stops, a notification lands in someone's inbox. The cost is short-lived and bounded.
A silent failure has none of these signals. Nobody gets an email, the dashboard looks normal, the log is full of "success". Meanwhile the damage grows linearly with time — every day adds another batch of unprocessed orders or a few more records with the wrong value. Worse, the cost of the fix grows too: repairing three days of data is a manual afternoon, repairing three months of data is a project with its own budget, a migration script and the need to explain to customers why they received the wrong billing.
An automation that fails loudly costs you hours. An automation that fails silently costs you weeks.
The practical consequence is simple: monitoring should not verify that the process runs. It should verify that the process still does what you expect it to do.
Five ways automations actually fail
Most real-world outages fall into a handful of recurring patterns. It pays to walk through them when designing every scenario and ask what happens if each one occurs.
| Failure mode | How it shows up | Typical way to detect it |
|---|---|---|
| External API change or outage | 4xx/5xx errors, empty responses, renamed fields | Call error rate, response schema check |
| Credential or token expiry | Sudden 401/403 after months of clean runs | Alert on authorization errors, expiry tracking |
| Unexpected input format | A new column in an export, a different decimal separator, an empty field | Input validation, count of rejected records |
| Partial completion | The first step ran, the second did not — data left inconsistent | Reconciliation check between systems |
| Silent success with wrong data | Everything "passed", but the values are wrong | Business-level check of the result, deviation from expectation |
The last row is the nastiest. If net and gross amounts get swapped in a field mapping, the system has no reason to report an error — it received a number and stored a number. Only a check that knows roughly what the number should be will catch it.
What to monitor beyond "it ran"
Most companies monitor at the level of "the scenario finished without an error". That catches maybe half of the real problems. A useful set of signals has four layers.
Volume against an expected baseline
If you normally get 200 to 400 orders a day and today the system processed eight, there may be no error anywhere — and yet something is fundamentally wrong. Set an expected range for each working day (and a separate one for weekends if they differ) and alert on values outside it. The upper bound matters just as much: double the usual volume usually means something got processed twice.
Run duration
A gradually lengthening run is an early warning sign. It signals growing data volume, a slowing database or repeated background retries. A run that has grown from two minutes to twenty has not failed yet — but it will, the moment it hits the platform's time limit.
Error rate, not error count
Five errors out of ten records is a disaster; five errors out of ten thousand records is normal operation. Track the ratio, not the absolute number, and set a threshold above which the alert escalates.
A business-level check of the result
This is the layer that is missing most often, and it is precisely the one that catches silent failures. Instead of a technical status it verifies the outcome: does the total of issued invoices match the orders for the same period? Does every new web contact have a matching CRM record? Has anything been stuck in an intermediate step for more than 24 hours? Such a check can be written as a separate daily scenario that changes nothing and only compares two sources — and it is usually the cheapest insurance in the whole system. If you already have reporting built on top of operational data, it is worth attaching these checks directly to it.
Idempotency: so a retry does not become a second failure
When monitoring catches a problem, the first instinct is "let's run it again". The question is whether you are allowed to. An idempotent process is one that, run repeatedly with the same input, ends in the same state — it does not create a second invoice, send a second email, or charge a payment twice.
In practice this means three things. First, every input item has a stable identifier (order number, record ID) that lets you check whether it has already been processed. Second, before an action is taken the system checks whether the output already exists — instead of "create an invoice", the rule is "create an invoice if one does not yet exist for this order". Third, steps with irreversible external effects (a payment, a sent message, a document handed over to accounting) are written to their own log before they execute, so that after a crash you can determine what actually went out.
Without this you get the worst possible state: you know the process failed, but not how much of it ran, so you are afraid both to re-run it and to fix it by hand. That is the moment an incident turns from an hour-long matter into a two-day one. When connecting several systems, this is the main argument for a well-designed data exchange — a topic we cover in detail in the article on connecting company systems via API.
Alerts, a named owner and alert fatigue
An alert that lands in a shared inbox or a channel with 40 people is not an alert. It is information everyone reads on the assumption that someone else is handling it. A working setup always has one specific name attached.
| Severity | Example situation | Where it goes | Expected response |
|---|---|---|---|
| Critical | Payments or invoicing did not process; data left inconsistent | Phone/SMS to the process owner | Within an hour, including outside working hours |
| High | The scenario did not run, volume outside the expected range | Direct message to the owner | Same working day |
| Low | Individual rejected records, lengthening run duration | Daily digest to the team channel | Within a week, as part of maintenance |
Alert fatigue is a real risk and it always arises the same way: the threshold is set too sensitively, people start ignoring alerts, and on the day the important one arrives it passes unnoticed. The remedy is not to blindly send fewer alerts, but to split them by severity and ruthlessly retire the ones nobody ever acted on. An alert that does not lead to an action must either be fixed or switched off.
And the single most important rule in this article: every automated process has a named human owner. Not "IT", not "the vendor", but a specific person who knows what the process does, what happens if it does not run, and who has the authority to decide on switching to manual mode. When people leave the company, ownership has to be handed over explicitly — otherwise you end up with orphaned automations that run for years without anyone understanding them. The same applies to AI components; the line between fully autonomous operation and agents with human oversight is essentially a decision about where a human stays in the loop.
The manual fallback and a post-incident habit
One day the automation will not run. The question is not whether, but what happens to the work it was supposed to do. The answer should be written down — one page, stored outside the system that may itself be unavailable, and accessible to the people who will have to deal with it.
A useful fallback procedure covers: who decides to switch to manual mode, how the process is carried out by hand step by step, where the list of manually processed items is collected (so they are not processed a second time after recovery), what outage length is acceptable, who to escalate to, and how you verify after the automation is restored that nothing is missing and nothing is duplicated.
A written procedure nobody has ever tried is fiction. A short rehearsal once a year — half an hour, on test data — reliably reveals that only a former colleague has the access credentials, or that the "manual export" takes three hours instead of ten minutes. It is equally worth building a post-incident habit: a short note on what happened, how long it lasted, why nobody noticed sooner, and which new signal was added so that next time it surfaces faster. Without that last question the same failure will repeat in a different process. This discipline belongs to ordinary software maintenance and development after launch, not to crisis mode.
The choice of tool matters less here than it seems. Platforms like Make or Zapier have basic notifications about failed runs, but you still have to design the business-level checks and idempotency yourself — you will find a comparison of the approaches in the article on workflow automation using ready-made platforms versus a custom solution. With a custom build you get more control over logging and retry logic, but also more responsibility for making sure someone actually builds it.
When is all of this unnecessary? If the process runs once a month, someone reviews its output anyway and a mistake can be fixed in ten minutes, a simple failure notification is enough. The full treatment is deserved by processes that touch money, customer communication, or data another system relies on.
Summary
Automation reliability does not come from the scenario being written correctly, but from its failure being visible, repeatable and owned by someone. Track volume, duration, error rate and at least one business-level check of the result. Design processes so they can be safely re-run. Route alerts to a specific person and distinguish their severity. And keep a written manual procedure that the team tries out from time to time.
If you are deploying automations or AI agents and process automation and want to review where a silent failure would do the most damage, get in touch with us — we will go through your processes and propose monitoring and fallback plans matched to their real impact.