Knowing a flow did the right thing

Every step returned a 200. The flow was green. It had written eleven empty rows.

Automation platforms report on execution. Did each step run, did anything throw, did the run finish. These are useful facts and they are not the fact you care about, which is whether the thing you wanted to happen happened.

The gap between them is where the expensive failures live. An API that changes its response shape returns 200 with a field you no longer read. A filter that matches nothing produces an empty list, and an empty list is a perfectly valid input to the next step. A date parsed in the wrong format is still a date. Nothing errors, so nothing is reported.

Borrowing an old idea

Test suites solved this decades ago with the assertion: a statement about what should be true at a given point, which fails the run when it is not. There is no reason that idea should stay inside test suites. A scheduled workflow is a program that runs unattended, which is exactly the situation where you least want to rely on somebody eyeballing the output.

So we added an assert step. It sits anywhere in a flow, examines a value produced by an earlier step, and either passes silently or stops the run with a message you wrote.

fetch orders          ->  213 records
assert  count > 0      ->  pass
transform             ->  213 rows
assert  no empty email ->  FAIL: 4 rows missing email
                          run stopped, alert sent

Where to put them

Assertions are cheap, but scattering them everywhere makes flows tedious to read. Three positions earn their place.

Immediately after anything you did not write

Every external API is a thing that can change without telling you. An assertion just after the call is the difference between finding out on the day it changes and finding out when somebody downstream complains about the numbers.

Before anything irreversible

Sending email, posting to a channel, writing to a system of record, charging somebody. The step before an irreversible action is the last moment where stopping is free. This is the single highest-value place to assert, and the one most often left empty.

At the end, about the outcome

Not "did the steps run" but "does the world now look how it should". Row count matches what came in. The file exists. The total balances. This is the assertion that catches the failure nobody predicted, because it describes the goal rather than the mechanism.

What a good failure message looks like

The assertion is only half the value. The other half is the sentence it leaves behind when it fails, and that sentence is read by you, six weeks later, at an inconvenient hour, with no memory of building the flow.

• Weak: assertion failed
• Better: expected more than 0 orders, got 0
• Best: expected more than 0 orders, got 0 — upstream filter probably matched nothing after the status values changed

The third one contains a hypothesis. You were the person best placed to guess the cause at the moment you wrote the check, and that guess is worth writing down.

The trade

Assertions stop flows. That is the point, and it is also the cost: a flow that halts on a condition you specified too tightly is a flow that did not do its job. A count assertion with a floor set at last week's volume will fire on a quiet Monday.

Set bounds you would genuinely want to be woken for. An assertion that cries wolf gets removed, and a removed assertion protects nothing.