# dumortier-marketing-ai-os-verification-2026-08-12

## Veille

Experience report published on **LinkedIn Pulse** on **August 12, 2026** by **Guillaume Dumortier**, in his newsletter *Growth Marketing Fit*, subtitled *« Four layers, a lot of rebuilding, and the failure modes nobody warns you about »*, ~2,500 words. The subject: an internal AI system built **in Claude** for a marketing team of about sixty people — roughly thirty content and sales **skills**, a dozen **source-of-truth modules**, **seven agents, six of which exist only to check work rather than produce it**, a **plugin** for those who live in a terminal, a **browser application** carrying the same knowledge for everyone else, and an orchestration that chains three or four assets into a *campaign bundle*. The thesis is set out early: the quality of an AI output is not determined at the moment of generation, but by what the system knows before it starts and by what happens to the draft afterward — *« The generation step in the middle is the easy part. It's also the only part most teams have built. »* Hence four layers: **Truth** (almost nobody builds it), **Production** (everybody), **Verification** (almost nobody), **Internal distribution** (*« where good systems die of neglect »*). Two failure mechanisms carry the article. **(A) The verifier's bare closed-world « pass »**: a fact-checker backed by product documentation receives a draft containing a claim about another product, one its sources did not cover — it returns a *« pass »*, not because the claim was true but because nothing contradicted it. *« It didn't just miss the error, it certified it. »* Fix: forbid a bare verdict and require every report to declare its **own coverage** — how many claims were checked, how many matched to sources, which fell outside its jurisdiction, which were owned by no source. *« "I can't verify this" became a first-class result. »* **(B) The cross-asset contradiction**: two assets can each be individually correct, each traceable to a real source, and still contradict each other — the press release states one date, the blog post another, both pass, the bundle can't ship. *« Per-asset verification can't catch that, by construction. »* Article's closing clause: *« The generation is free. The trust is the product. »*

## Titre Article

I built a marketing AI operating system for a 60-person team. The most valuable thing in it is the part that refuses to write.

## Date

2026-08-12

## URL

https://www.linkedin.com/pulse/i-built-marketing-ai-operating-system-60-person-team-most-dumortier-3hbnc

## Keywords

Guillaume Dumortier, Growth Marketing Fit, LinkedIn Pulse, marketing AI OS, AI marketing, internal tool, Claude, skills, plugin, browser application, four layers, truth layer, truth layer, source of truth, document owner, versioning, provenance, production layer, verification layer, internal distribution, internal adoption, trust machine, fact drift, facts/content separation, skill routing, skill boundaries, output contract, ambiguity, outline instead of article, hollow review, rubber-stamping, output laundering, fresh session, closed world, closed-world, coverage statement, bare-pass ban, unverifiable claim, first-class result, campaign bundle, cross-asset verification, cross-consistency, contradiction between assets, review that never blocks, two interfaces, adoption follows trust, visible uncertainty, deterministic control, enforcement in code, em dash, asking nicely is not a control, silent failure, empty constant, number stripping, misattributed hallucination, pipeline test, validation gap, refusal, teach the system to refuse, verification cost, staleness, ownership and provenance

## Authors

**Guillaume Dumortier** — auteur de la newsletter LinkedIn **Growth Marketing Fit** (~1 300 abonnés à la publication). Il écrit en **praticien-constructeur** : il a passé *« une longue partie de cette année »* à bâtir et exploiter le système décrit. La légende de l'illustration précise le socle technique — *« A custom-built Marketing AI OS within Claude »*. Publié le **12 août 2026**.

## Ton

**Profile**: internal engineering experience report, first person, published as a LinkedIn newsletter. Audience bounded from the introduction: *« This is not a piece about whether AI can write well. If that's still the open question for you, this won't land. It's for people already three months in, quietly wondering why the thing works beautifully in demo and keeps producing outputs nobody trusts enough to ship. »* Anti-demo practitioner register, dry, without salesperson vocabulary.

**Style**: the structure is fixed and repeated for each layer — six headings in the same order, four times: *What it is* → *Why nobody builds it* → *What I got wrong* → *The rule* → *How you'll know it's working* → *Do this week*. It's an audit template that can be run against one's own system. Success criteria are phrased as observable signals rather than metrics: *« Someone asks "where did this number come from" and the answer takes four seconds instead of an archaeological dig through Slack »*, *« people stop asking you which tool to use for what »*. The text never prescribes without first recounting the corresponding failure, admitted in ridiculous detail — the capital letters in the prompt (*« which is roughly the point you should start suspecting you have the wrong model of the problem »*), the blog skill that described articles instead of writing them, *« every single time »*, for weeks. Every section closes on an aphorism, which makes the text easy to quote without the underlying mechanism. The *« what I'm still not sure about »* section is placed before the conclusion and not shied away from, with self-criticism: *« I currently do a mix and I don't think the mix is reasoned, I think it's a tradeoff I haven't properly examined. »*

**Marker phrases**:
- ***« I was building a trust machine, and I didn't know it »***
- ***« You cannot demo a truth layer. You can only demo what it prevents, which is nothing visible »***
- ***« Nothing that produces content is allowed to contain a fact. It has to ask »***
- ***« a skill's most important job is describing what it is not for »***
- ***« You do not get a bad review. You get a good review that is worthless »***
- ***« It didn't just miss the error, it certified it »***
- ***« An unverifiable claim is a finding, not a silence »***
- ***« If your review step has never blocked anything, it isn't a review step. It's decoration »***
- ***« adoption tracks trust, not capability »***
- ***« Never ask a model for something you can enforce in code »***
- ***« Traditional software crashes when it breaks. These systems keep going, confidently, at reduced quality »***
- ***« The generation is free. The trust is the product »***

**Epistemic position**: practitioner in operation, not in demonstration. The article names neither the company, nor the products, nor usage figures: no result is measured, no adoption quantified, no before/after produced — which the author claims openly (*« I'm going to tell you what broke, because the broken parts are the useful part »*). Strong authority on the failure mechanisms encountered; none on their frequency, cost, or generality. Limits to keep in mind: n=1, a single stack, several fixes generalized by the author without demonstration, and an acknowledged but unquantified cost.

## Pense-betes

- **Date / source**: **August 12, 2026**, LinkedIn Pulse, newsletter *Growth Marketing Fit*, ~2,500 words. System built in **Claude** for a marketing team of ~60 people.
- **Key framing**: *« I thought I was building a content machine. I was building a trust machine, and I didn't know it, so I spent my initial effort in exactly the wrong place. »* ### Forbid the bare « pass » A verifier is a **closed-world system**: it can only rule on what it's been given. Faced with a claim none of its sources cover, it finds no contradiction and returns a favorable verdict indistinguishable from a real check. The fix is a format constraint: | The report must state | Why | |---|---| | How many claims were **checked** | denominator: without it, an « OK » means nothing | | How many were **actually matched to a source** | that's the true coverage rate, always lower | | Which ones **fell outside its jurisdiction** | otherwise the judge overreaches and other claims' silence passes for agreement | | Which ones are **owned by no source in the system** | that's the organization's real risk register | *« I can't verify this »* must be a first-class result, on the same footing as « pass » and « fail ». The symmetric anti-pattern is more dangerous than no check at all: *« a good review that's worthless is far worse than no review, because it launders the output »* — someone downstream sees *reviewed: pass* and stops looking. Cross-reference [[willison-fable-judgement-delegation-subagents-2026-07-03]]. ### The check that per-asset verification can't produce Two assets from the same campaign can each be individually correct, each traceable to a real source, each validated — and still contradict each other. Verifying against the truth layer and verifying assets against each other are **two different checks**; the first can't produce the second. The author's diagnosis: *« if you run multi-asset campaigns and only verify one asset at a time, you have this bug right now. »* Transposition outside marketing: a PR touching three files, a batch-generated set of specs, a doc and its code produced together. Cross-consistency is a bundle-level check, never a sum of unit checks. ### The truth layer Failure recounted: product facts lived **inside** the skill that wrote the articles, then the same fact had to exist in the email skill, the battlecard, the web page. *« Within a few weeks I had four slightly different versions of our launch date in four different files, and the drift was invisible because each file was individually plausible. »* Rule: *« a document that states facts and a document that produces content are two different documents, with two different owners »*, each versioned and dated. Nothing that produces content is allowed to contain a fact — it has to ask for it. Benefit: a fact changes, you change it once, all 35 skills are correct the next day; a claim is disputed, there's only one place to look. Why nobody builds it: *« You cannot demo a truth layer. You can only demo what it prevents, which is nothing visible. »* Proposed exercise: open the last five assets produced, highlight every factual claim, name for each one the single document that owns it — *« the ones you can't assign an owner to are your real risk register. »* Extends [[vasilopoulos-codified-context-infrastructure-ai-agents-2026-02-24]]. ### Two lessons about skills 1. **The output contract has to be paranoid about ambiguity.** For weeks, the « blog post » skill wrote **descriptions of articles** — section headers followed by a sentence explaining what the section would cover, *« every single time »* — and it passed every review, because the structure was flawless. Cause: an ambiguous line in the template (*« the post, under the flavour's own section headers »*), reasonably read as a request for an outline. Organizational detail more important than the bug: nobody flagged it, *« people assume the tool is right and they're using it wrong. »* 2. **A skill's most important job is describing what it is not for.** Past about thirty skills, the problem stops being quality and becomes **routing**: two skills that both plausibly handle *« write me something for sales »* fight over every request, and the arbitrary winner produces the wrong format. Half of every description became an explicit boundary. Compare with [[shihipar-claude-code-lessons-building-skills-2026-06-03]]. ### From prompt to plumbing Voice rules forbade em dashes, every prompt said so, testers still found them for weeks: *« I kept repeating the rule louder, which is a probabilistic fix to a problem whose deterministic solution was right there. »* Fix: a single output-side function that strips them. Generalizable heuristic: *« if you catch yourself repeating an instruction, that's the signal to pull it out of the prompt and put it in the plumbing »*, with its corollary — *« asking nicely is not a control. »* ### The silent failure A constant meant to hold an invisible marker character had been emptied to a blank string. No visible diff, no error. Downstream, a cleanup step started matching every digit in every prompt and stripping it: `« 7.1% across 6,000+ orgs, 2500 words »` reached the model as `« .% across ,+ orgs, words »`. Every statistic, date, word count, and section number silently removed, for an unknown number of releases. That was the real reason cited figures kept coming back wrong — *« and I'd spent weeks blaming the model's tendency to make up numbers. It was making them up because I'd deleted them. »* Principle: *« Traditional software crashes when it breaks. These systems keep going, confidently, at reduced quality, and produce something that looks correct. »* Two fixes: test **the pipeline**, not just the output — a test whose sole function is to assert that numbers survive a round trip through prompt assembly; and remember that validation has the same gaps as the system — a field capped at 500 characters stayed at 688 for two releases because the script checked every other limit except that one. Diagnostic rule: before accusing a model of inventing a number, check that the number actually reached it. ### Layer 4, internal distribution *« This is where most internal AI projects die quietly. Not from technical failure. From being technically excellent and used by four people. »* The author had built for himself first — a plugin requiring command-line fluency, which described six people out of sixty. Fix: the same system built twice, same knowledge and same checks, two entry points — a plugin for builders, a browser application for everyone else (catalog, three fields, a draft with its checks alongside as buttons). The finest point in the section: **adoption follows trust, not capability**. The tool got more use once the output started admitting what it wasn't sure of — *« A draft that flags "this customer example is illustrative, find a real one before publishing" gets used. A draft that confidently presents a made-up customer example gets used once, embarrasses someone, and the tool dies by word of mouth. »* ### Teach the system to refuse The last piece adapts an asset for another market segment and knows the segment the company decided not to pursue: asked for that segment, it declines and explains why. *« Same principle as the coverage statement. A system that can only say yes will confidently hand you the wrong thing forever, and you won't be able to tell "this is right" from "this was the only answer available." »* Refusal and admitting ignorance are the same primitive: the system's ability to bound its own domain of competence. ### The three uncertainties left open 1. **Embedded truth layer or fetched live?** *« Embedded copies go stale without anyone noticing. Live fetches are slow and break when someone renames a folder. »* The author does a mix and says it isn't a reasoned one. 2. **Does the verification cost still hold up?** A fully checked asset costs *« several times »* a raw draft: worthwhile for public-facing work, probably not for an internal summary, *« and the line between the two is blurrier than my system claims. »* Governance variable to instrument first: a verification level tied to asset criticality. 3. **How much of this survives the next two generations of models?** His bet, presented as such: the truth layer and the coverage discipline survive, because they *« solve an organizational problem of provenance and ownership that would exist even with a perfect model. »* ### The « if you're starting Monday » plan 1. Write down the **ten facts** the team repeats most, with an owner and a date for each — truth layer version zero, *« it takes an afternoon. »* 2. Take your best prompt and pull every fact out of it into that document; make the prompt ask for them. 3. Build a verification step that runs **in a fresh session**, sees only the draft and the sources, and must state what it couldn't check. 4. Find what keeps getting repeated in prompts and move it into code. 5. Show it to a non-technical person and watch them use it without helping. Baseline test to run before anything else: run an asset that's **already shipped** through the check in a completely fresh session, with no context but the draft and the sources. *« What comes back is your real quality baseline. It's usually humbling. Mine was. »*

## RésuméDe400mots

Experience report published on **LinkedIn Pulse** on **August 12, 2026** by **Guillaume Dumortier** (newsletter *Growth Marketing Fit*), on an internal marketing AI system built **in Claude** for a team of about sixty people: roughly thirty skills, a dozen truth modules, **seven agents, six of which only check work**, a terminal plugin, a browser application, and multi-asset campaign orchestration.

**The thesis.** *« I thought I was building a content machine. I was building a trust machine. »* The quality of an AI output is not determined at generation, but by **what the system knows beforehand** and **what happens to the draft afterward**. Generation is the easy part — and the only part most teams have built.

**Four layers.** *Truth*: fact documents separated from anything that produces content, each with an owner, versioned and dated. Leaving facts inside the skills produced **four versions of a launch date across four files**, each individually plausible. *Production*: the blog skill spent weeks writing **descriptions of articles** instead of articles, and passed every review, because the review checked the structure. Past thirty skills, the problem becomes **routing** — half of each skill description has to state what it's not for. *Verification*: the layer that separates a demo from a system. *Internal distribution*: where projects die from being excellent and used by four people.

**The two central failures.** A fact-checker receives a claim none of its sources cover: it returns a « pass ». *« It didn't just miss the error, it certified it. »* Fix: a verifier is a **closed-world system**; **it is forbidden from returning a bare « pass »** and must declare its coverage — how many claims checked, how many actually matched, which fell outside its jurisdiction, which were owned by no source. *« An unverifiable claim is a finding, not a silence. »* Second failure: **two individually correct assets can contradict each other**; per-asset verification can't catch it, by construction.

**Five cross-cutting rules.** Never ask a model for something you can enforce in code. **Silent failures** are the whole risk — an emptied constant stripped every number from every prompt, and it blamed the model for hallucinating. Test the pipeline, not just the output. Your validation has the same gaps as your system. **Teach the system to refuse.**

**Adoption follows trust, not capability**: an output that admits what it's unsure of gets used. Closing clause: ***« The generation is free. The trust is the product. »***

## GrapheDeConnaissance

- Guillaume Dumortier —a_créé→ Marketing AI OS (TECHNOLOGIE, 0.96)
- Marketing AI OS —utilise→ Claude (TECHNOLOGIE, 0.93)
- Marketing AI OS —est_instance_de→ système interne à quatre couches — vérité, production, vérification, distribution — servant une équipe marketing d'une soixantaine de personnes avec une trentaine de skills, une douzaine de modules de source de vérité et sept agents dont six ne font que contrôler (AFFIRMATION, 0.94)
- Guillaume Dumortier —affirme_que→ la qualité d'une sortie IA n'est pas déterminée au moment de la génération mais par ce que le système sait avant de commencer et ce qui arrive au brouillon après qu'il a fini (AFFIRMATION, 0.95)
- Guillaume Dumortier —affirme_que→ il croyait construire une machine à contenu alors qu'il construisait une machine à confiance, ce qui lui a fait dépenser son effort initial au mauvais endroit (CITATION, 0.95)
- couche de vérité —fait_partie_de→ Marketing AI OS (TECHNOLOGIE, 0.94)
- couche de vérité —réduit→ la dérive factuelle : un fait modifié une seule fois rend correctes les trente-cinq skills dès le lendemain, et toute affirmation contestée n'a qu'un document propriétaire et qu'une personne à interroger (AFFIRMATION, 0.92)
- Guillaume Dumortier —recommande→ qu'un document qui énonce des faits et un document qui produit du contenu soient deux documents différents avec deux propriétaires différents, rien de ce qui produit du contenu n'ayant le droit de contenir un fait (AFFIRMATION, 0.95)
- vérification en monde clos —s_applique_à→ tout vérificateur automatique adossé à un corpus de sources fini, qui ne peut se prononcer que sur ce qu'on lui a fourni (AFFIRMATION, 0.93)
- vérification en monde clos —observé_dans→ un vérificateur de faits ayant rendu un « pass » sur une affirmation portant sur un produit que ses sources ne couvraient pas : rien ne la contredisait, donc il n'a trouvé aucun problème et a certifié l'erreur au lieu de la manquer (AFFIRMATION, 0.95)
- déclaration de couverture —résout→ la certification en monde clos : le vérificateur ne peut pas renvoyer un « pass » nu et doit énoncer combien d'affirmations il a contrôlées, combien il a réellement appariées à ses sources, lesquelles ne relevaient pas de sa juridiction et lesquelles ne sont possédées par aucune source du système (AFFIRMATION, 0.94)
- Guillaume Dumortier —affirme_que→ une affirmation invérifiable est un constat et non un silence, « je ne peux pas vérifier ceci » devant être un résultat de première classe (CITATION, 0.95)
- Guillaume Dumortier —affirme_que→ une bonne revue sans valeur est bien pire que pas de revue du tout, parce qu'elle blanchit la sortie et que quelqu'un en aval voit « reviewed: pass » et arrête de regarder (AFFIRMATION, 0.94)
- vérification inter-actifs —résout→ la contradiction entre actifs d'un même lot : deux actifs peuvent être individuellement corrects, traçables vers de vraies sources et validés, tout en se contredisant, ce que la vérification par actif ne peut pas attraper par construction (AFFIRMATION, 0.94)
- Guillaume Dumortier —recommande→ que chaque contrôle tourne comme un appel réellement séparé, sans mémoire de la rédaction du brouillon, différents contrôles possédant des juridictions disjointes et n'ayant pas le droit de noter le terrain des autres (AFFIRMATION, 0.93)
- Guillaume Dumortier —affirme_que→ le travail le plus important d'une skill est de décrire ce à quoi elle ne sert pas, le problème cessant d'être la qualité pour devenir le routage passé une trentaine de skills (AFFIRMATION, 0.94)
- ambiguïté du contrat de sortie —observé_dans→ une skill d'article de blog ayant produit pendant des semaines des descriptions d'articles au lieu d'articles, à cause d'une seule ligne ambiguë du gabarit, et passant toutes les revues parce que la revue contrôlait la structure (AFFIRMATION, 0.93)
- Guillaume Dumortier —recommande→ de ne jamais demander à un modèle ce qu'on peut imposer en code, la répétition d'une instruction dans les prompts étant le signal de la déplacer dans la plomberie — demander gentiment n'est pas un contrôle (CITATION, 0.95)
- échec silencieux —observé_dans→ une constante censée contenir un caractère marqueur invisible vidée en chaîne vide, faisant supprimer par une étape de nettoyage chaque chiffre de chaque prompt pendant un nombre indéterminé de releases, sans diff visible ni erreur (AFFIRMATION, 0.95)
- Guillaume Dumortier —affirme_que→ le logiciel traditionnel plante quand il casse alors que ces systèmes continuent avec assurance à qualité réduite et produisent quelque chose qui a l'air correct (CITATION, 0.94)
- Guillaume Dumortier —recommande→ de tester le pipeline et pas seulement la sortie, par exemple un test dont l'unique fonction est d'affirmer que les chiffres survivent à un aller-retour dans l'assemblage du prompt (AFFIRMATION, 0.93)
- Guillaume Dumortier —affirme_que→ l'adoption suit la confiance et non la capacité : un brouillon qui signale son incertitude est utilisé, un brouillon qui présente avec assurance un exemple inventé est utilisé une fois puis l'outil meurt par le bouche-à-oreille (AFFIRMATION, 0.93)
- Guillaume Dumortier —recommande→ de construire la seconde interface plus tôt que ce qui semble justifié et de rendre l'incertitude du système visible plutôt que de la cacher pour paraître plus impressionnant (AFFIRMATION, 0.92)
- Guillaume Dumortier —recommande→ d'apprendre au système à refuser, une étape déclinant explicitement une demande hors périmètre commercial, parce qu'un système qui ne peut que dire oui tendra avec assurance la mauvaise chose sans qu'on puisse distinguer « c'est juste » de « c'était la seule réponse disponible » (AFFIRMATION, 0.94)
- Guillaume Dumortier —affirme_que→ une étape de revue qui n'a jamais rien bloqué n'est pas une étape de revue mais de la décoration (CITATION, 0.94)
- Guillaume Dumortier —mesure→ un actif entièrement vérifié coûte plusieurs fois le prix d'un brouillon brut, ce qui est justifié pour toute publication publique mais probablement pas pour une synthèse interne (MESURE, 0.85)
- Guillaume Dumortier —prédit→ que la couche de vérité et la discipline de couverture survivront aux deux prochaines générations de modèles, parce qu'elles ne compensent pas un raisonnement faible mais résolvent un problème organisationnel de provenance et de propriété qui existerait même avec un modèle parfait (AFFIRMATION, 0.9)
- Guillaume Dumortier —affirme_que→ la génération est gratuite et que la confiance est le produit (CITATION, 0.96)
- déclaration de couverture —s_applique_à→ tout dispositif de LLM-juge hors marketing, notamment la revue de code automatisée et l'évaluation de sorties générées (AFFIRMATION, 0.85)
- Growth Marketing Fit —publie→ Marketing AI OS (TECHNOLOGIE, 0.85)

---
Canonical: https://www.thekb.eu/en/fiches/dumortier-marketing-ai-os-verification-2026-08-12/