The intake pipeline
Six stages from a raw Messenger export to knowledge-base documents. Half of them make no model calls at all.
Normalize and assign provenance
The export becomes records of a conversation id, its timestamps, a cohort, and its turns. Each turn keeps only who spoke, when, its type, and its text.
Attachment URLs are dropped: they are signed, expiring CDN links that cannot be
resolved later and would pull customer-uploaded images into the repo. A turn carrying
no text becomes a placeholder such as <image>, preserving the one
thing extraction needs — that a turn happened and carried nothing readable.
Cohort is decided by where a conversation starts, never by cutting it at the date. Truncating a straddling thread would leave the surviving half reading as a reply to turns the extractor cannot see, and the turns removed would be exactly the bot-era ones the cutoff exists to exclude. 135 conversations showing a bot marker are excluded entirely.
Triage
Every conversation is labelled substantive, deflection, lead-capture, boilerplate-only or no-agent-reply. Nothing is dropped; the purpose is an auditable denominator for the results.
Canned templates are found by counting identical agent messages across the whole export, including the bot era — 47 of them. Deriving that from the post-cutoff window alone would miss almost every template, because a message sent 128 times over eight months may appear only once inside a single month.
Extract atomic claims
One call per conversation. Each yields self-contained factual statements tagged with the subject they concern, the kind of knowledge they are, and the cohort inherited from their source.
The prompt carries a section the call-transcript version never needed: what is not a claim. Canned greetings, requests for a callback number, promises that staff will follow up, and bare links are all excluded by name.
Each fact also records whether its subject was stated in the same message or inferred from context. Agents routinely quote a bare price with no product named, so an inferred subject on a price is the pipeline’s main source of wrong facts. 33 of 131 facts are inferred, including 11 of the 21 prices.
Of 160 claims extracted, 29 were about one customer only and were dropped; the privacy gate withheld none.
Review the plan
Before anything is written, the full plan is emitted with every claim visible — which subjects become documents, what each will contain, its provenance mix, which facts rest on an inferred subject, and which subjects fall below the threshold. Grouping mistakes are cheap to fix here and expensive afterwards.
This stage also resolves every currency amount claimed by more than one product, ranking the claimants by how many separate conversations assert the figure, then by how many named the product outright. Losing claims are dropped here, before the size threshold, and every one is written to an audit file with the basis for the decision.
Consolidate subjects
Extraction names subjects freely, so the same product arrives under many labels — 64 of them. One call consolidates these into 26 canonical subjects, each a candidate document.
It works from evidence rather than wording: every label is presented with sample claims and with the labels it co-occurs with. Unlike the call POC it receives no list of product names — the chat export has no per-customer interest field to build one from — so the sample claims are its only evidence for what a product is called.
Synthesize documents
One call per section rather than per document. Each receives every claim for that section and writes the section body: grouping facts into tasks, and merging the many restatements of one fact into a single line.
Repetition is explicitly not treated as corroboration when it comes from a canned reply — one template hitting six conversations is one fact, not six.
A section quoting more than one amount, or containing any inferred-subject fact, receives a reconciliation instruction computed in code before the call is made: check whether the amounts can all be true, and prefer a stated subject over an inferred one. Leaving the writer to notice on its own did not work — two prices contradict each other only given a minimum-users fact that lands in a different section.
What each stage produced
| Stage | In | Out |
|---|---|---|
| Normalize | 246 conversations | 111 kept |
| Extract claims | 111 conversations | 131 facts |
| Consolidate subjects | 64 labels | 26 subjects |
| Resolve contested amounts | 2 collisions | 3 facts dropped |
| Synthesize | 109 facts | 15 documents |
The two cohorts
| Cohort | Conversations | Rule | Strength |
|---|---|---|---|
post-bot |
32 | Starts on or after 9 July 2026 | Strong — the generative bot had stopped |
no-bot-evidence |
79 | Starts earlier, carries no bot marker | Weak — markers cover only 37% of known-bot messages |
| excluded | 135 | Starts earlier, shows a bot marker | Not processed |
The second cohort is named for what it is. The supporting argument — that the greeting fires outside business hours, so a thread without one arrived while staff were on shift — is indirect, and three conversations disprove even that.
Personal data
Excluded at three points
The export pre-masks phone numbers, email addresses and identifiers at source, which is upstream of this pipeline and not relied on as complete. On top of that: attachment URLs are dropped during normalization; the extraction prompt forbids personal data by name; and a deterministic gate runs over both the fact and its subject — the subject becomes a document title and is prepended to every chunk of that document, so it reaches search results as surely as the fact does.
The company’s own published channels pass deliberately: a support knowledge base should contain them. On this corpus the gate withheld 0 of 131 facts.