Your privacy policy is part of the codebase

A legal page is a set of claims about what your code does. Code changes. The claims do not change with it.

We added a support chat to this site. It sends a question to a third party's model, which means a new processor handling user text — an obvious thing to disclose, and we disclosed it.

While making that edit I read the rest of the policy properly for the first time in weeks, and found three statements that were no longer true. Not one of them had been written dishonestly. Every one of them had been true when written.

How a policy goes false

The mechanism is dull, which is why it is so common. Someone writes a privacy policy early, when the product is small and they know every line of it. The description is accurate. Then features ship. Each feature is a small change to what data is touched, and no individual change feels like a legal event. The policy is a static file nobody has a reason to open.

Six months later the document describes a product that no longer exists.

• A claim that a certain kind of data is never stored, written before a feature that stores it for debugging.
• A list of subprocessors that predates the two most recent integrations.
• A retention period that reflects the original design and not the current one.
• A statement about where processing happens, written before a regional restriction was added.

None of these is a lie. All of them are wrong, and wrongness is the thing that matters to a regulator, a customer's security review, or anyone deciding whether to trust you.

Treating it as a test problem

The realisation worth having is that these statements are testable. A sentence that says the service does not store credentials is a claim about the code. If the code stores credentials, a test can catch the contradiction — not by understanding English, but by checking that specific assertions in the document still line up with specific facts about the source.

So we wrote a small suite that runs with everything else:

legal drift
  policy names every third-party processor found in source ... ok
  policy does not claim we skip data the executor records  ... ok
  regional restriction in policy matches the guard in code ... ok
  retention statement matches the configured window        ... ok

These are crude checks. A test cannot tell you your policy is well written, and it cannot read intent. What it can do is fail loudly the day someone adds a fourth processor without touching the document — which is precisely the day the drift starts.

• TriggerNew outbound host appears in source, policy processor list unchanged
• TriggerExecutor begins persisting a field the policy says is not retained
• TriggerA region guard is added or removed without a matching disclosure
• Not coveredTone, completeness, jurisdiction-specific wording, anything requiring judgement

The awkward part

There is an uncomfortable inversion here. Most engineering effort goes into making the product do what the documentation promises. This is the reverse: making the documentation admit what the product actually does. It is less satisfying and more consequential, because the failure mode is not a broken feature. It is a written, published, signed statement that is untrue.

What we do now

The policy is versioned in the same repository as the code, which it always was, but it is now reviewed on the same trigger: any change to what leaves our servers, what we keep, or where it runs. The drift suite runs on every push. It has already caught one thing.

Legal pages get written by people being careful, and then decay because nobody is being careless. That is what makes them worth automating. The failure needs no villain.