Marketing AI OS: the Truth Layer That Refuses

Key takeaways

1. Date / source. August 12, 2026, LinkedIn Pulse, newsletter Growth Marketing Fit, ~2,500 words. System built in Claude for a marketing team of ~60 people.
2. Key framing. « I thought I was building a content machine. I was building a trust machine, and I didn't know it, so I spent my initial effort in exactly the wrong place. » ### Forbid the bare « pass » A verifier is a closed-world system: it can only rule on what it's been given. Faced with a claim none of its sources cover, it finds no contradiction and returns a favorable verdict indistinguishable from a real check. The fix is a format constraint: | The report must state | Why | |---|---| | How many claims were checked | denominator: without it, an « OK » means nothing | | How many were actually matched to a source | that's the true coverage rate, always lower | | Which ones fell outside its jurisdiction | otherwise the judge overreaches and other claims' silence passes for agreement | | Which ones are owned by no source in the system | that's the organization's real risk register | « I can't verify this » must be a first-class result, on the same footing as « pass » and « fail ». The symmetric anti-pattern is more dangerous than no check at all: « a good review that's worthless is far worse than no review, because it launders the output » — someone downstream sees reviewed: pass and stops looking. Cross-reference [[willison-fable-judgement-delegation-subagents-2026-07-03]]. ### The check that per-asset verification can't produce Two assets from the same campaign can each be individually correct, each traceable to a real source, each validated — and still contradict each other. Verifying against the truth layer and verifying assets against each other are two different checks; the first can't produce the second. The author's diagnosis: « if you run multi-asset campaigns and only verify one asset at a time, you have this bug right now. » Transposition outside marketing: a PR touching three files, a batch-generated set of specs, a doc and its code produced together. Cross-consistency is a bundle-level check, never a sum of unit checks. ### The truth layer Failure recounted: product facts lived inside the skill that wrote the articles, then the same fact had to exist in the email skill, the battlecard, the web page. « Within a few weeks I had four slightly different versions of our launch date in four different files, and the drift was invisible because each file was individually plausible. » Rule: « a document that states facts and a document that produces content are two different documents, with two different owners », each versioned and dated. Nothing that produces content is allowed to contain a fact — it has to ask for it. Benefit: a fact changes, you change it once, all 35 skills are correct the next day; a claim is disputed, there's only one place to look. Why nobody builds it: « You cannot demo a truth layer. You can only demo what it prevents, which is nothing visible. » Proposed exercise: open the last five assets produced, highlight every factual claim, name for each one the single document that owns it — « the ones you can't assign an owner to are your real risk register. » Extends [[vasilopoulos-codified-context-infrastructure-ai-agents-2026-02-24]]. ### Two lessons about skills 1. The output contract has to be paranoid about ambiguity. For weeks, the « blog post » skill wrote descriptions of articles — section headers followed by a sentence explaining what the section would cover, « every single time » — and it passed every review, because the structure was flawless. Cause: an ambiguous line in the template (« the post, under the flavour's own section headers »), reasonably read as a request for an outline. Organizational detail more important than the bug: nobody flagged it, « people assume the tool is right and they're using it wrong. » 2. A skill's most important job is describing what it is not for. Past about thirty skills, the problem stops being quality and becomes routing: two skills that both plausibly handle « write me something for sales » fight over every request, and the arbitrary winner produces the wrong format. Half of every description became an explicit boundary. Compare with [[shihipar-claude-code-lessons-building-skills-2026-06-03]]. ### From prompt to plumbing Voice rules forbade em dashes, every prompt said so, testers still found them for weeks: « I kept repeating the rule louder, which is a probabilistic fix to a problem whose deterministic solution was right there. » Fix: a single output-side function that strips them. Generalizable heuristic: « if you catch yourself repeating an instruction, that's the signal to pull it out of the prompt and put it in the plumbing », with its corollary — « asking nicely is not a control. » ### The silent failure A constant meant to hold an invisible marker character had been emptied to a blank string. No visible diff, no error. Downstream, a cleanup step started matching every digit in every prompt and stripping it: « 7.1% across 6,000+ orgs, 2500 words » reached the model as « .% across ,+ orgs, words ». Every statistic, date, word count, and section number silently removed, for an unknown number of releases. That was the real reason cited figures kept coming back wrong — « and I'd spent weeks blaming the model's tendency to make up numbers. It was making them up because I'd deleted them. » Principle: « Traditional software crashes when it breaks. These systems keep going, confidently, at reduced quality, and produce something that looks correct. » Two fixes: test the pipeline, not just the output — a test whose sole function is to assert that numbers survive a round trip through prompt assembly; and remember that validation has the same gaps as the system — a field capped at 500 characters stayed at 688 for two releases because the script checked every other limit except that one. Diagnostic rule: before accusing a model of inventing a number, check that the number actually reached it. ### Layer 4, internal distribution « This is where most internal AI projects die quietly. Not from technical failure. From being technically excellent and used by four people. » The author had built for himself first — a plugin requiring command-line fluency, which described six people out of sixty. Fix: the same system built twice, same knowledge and same checks, two entry points — a plugin for builders, a browser application for everyone else (catalog, three fields, a draft with its checks alongside as buttons). The finest point in the section: adoption follows trust, not capability. The tool got more use once the output started admitting what it wasn't sure of — « A draft that flags "this customer example is illustrative, find a real one before publishing" gets used. A draft that confidently presents a made-up customer example gets used once, embarrasses someone, and the tool dies by word of mouth. » ### Teach the system to refuse The last piece adapts an asset for another market segment and knows the segment the company decided not to pursue: asked for that segment, it declines and explains why. « Same principle as the coverage statement. A system that can only say yes will confidently hand you the wrong thing forever, and you won't be able to tell "this is right" from "this was the only answer available." » Refusal and admitting ignorance are the same primitive: the system's ability to bound its own domain of competence. ### The three uncertainties left open 1. Embedded truth layer or fetched live? « Embedded copies go stale without anyone noticing. Live fetches are slow and break when someone renames a folder. » The author does a mix and says it isn't a reasoned one. 2. Does the verification cost still hold up? A fully checked asset costs « several times » a raw draft: worthwhile for public-facing work, probably not for an internal summary, « and the line between the two is blurrier than my system claims. » Governance variable to instrument first: a verification level tied to asset criticality. 3. How much of this survives the next two generations of models? His bet, presented as such: the truth layer and the coverage discipline survive, because they « solve an organizational problem of provenance and ownership that would exist even with a perfect model. » ### The « if you're starting Monday » plan 1. Write down the ten facts the team repeats most, with an owner and a date for each — truth layer version zero, « it takes an afternoon. » 2. Take your best prompt and pull every fact out of it into that document; make the prompt ask for them. 3. Build a verification step that runs in a fresh session, sees only the draft and the sources, and must state what it couldn't check. 4. Find what keeps getting repeated in prompts and move it into code. 5. Show it to a non-technical person and watch them use it without helping. Baseline test to run before anything else: run an asset that's already shipped through the check in a completely fresh session, with no context but the draft and the sources. « What comes back is your real quality baseline. It's usually humbling. Mine was. »