Ofsted Signal Pipeline
Agentic pipeline · buying-intent signal
An unattended daily pipeline that turns the UK government's public Ofsted inspection feed into named, CRM-ready sales triggers — the day a provider is graded weak on exactly what Cognassist fixes.
In November 2025 Ofsted began publishing per-category inspection grades — including “Inclusion”, precisely what Cognassist sells into. Overnight the government started publishing a free, per-provider buying-intent signal, forever. But it ships as PDFs on a paginated site with no API: to use it you'd have to spot each new report, work out which legal entity it's about, pull three grades out of a PDF, match it to the CRM, qualify the provider, write a usable note, and judge whether it warrants a call — daily, across ~400 providers. A part-time job nobody had, and the signal decays to history in about six weeks.
Built a fully unattended pipeline that runs once a day on one machine and writes live to Capsule CRM with no human in the loop. The design bet: split the work by kind — deterministic Python for anything where a wrong answer corrupts data (identity resolution, grade extraction, CRM matching, all writes), an LLM agent for the reading and judgement (qualification, note-writing, signal-flagging). The agent orchestrates the scripts; it never re-implements them. Every column of the single 43-field dataset doubles as an idempotency key, so any stage re-runs safely. ~4,100 lines of Python, four dependencies, one CSV — no database.
Every morning it posts plain-English inspection summaries and outreach tasks straight into the CRM, and republishes one bookmarkable digest page. To date: 414 reports processed, UKPRNs resolved for 98.5%, 249 summary notes and 38 tasks written to Capsule. The commercial core is 38 named providers each carrying a public government document that says they're weak on exactly what Cognassist solves — already in the CRM, graded, with a task on someone's list.
Ofsted-Reports/ ├── CLAUDE.md # Agent context — read first ├── daily_pipeline_prompt.md # The instructions the agent follows │ │ ── DETERMINISTIC CORE · pipeline/ · 2,313 lines ── ├── ofsted_report_scraper.py # Stage 1 · incremental scrape + PDF (847) ├── enrich_ukprn.py # Stage 2 · URN → UKPRN, authoritative (222) ├── extract_grades.py # Stage 3 · 3 toolkit grades from PDF (243) ├── classify_capsule.py # Stage 4 · match provider to CRM (349) ├── add_capsule_prospects.py # Stage 5b · create prospects [LIVE] (440) ├── create_capsule_tasks.py # Stage 7 · signal → CRM task [LIVE] (262) │ │ ── JUDGEMENT LAYER · .claude/skills/ ── ├── provider-icp-qualifier/ # Rubric: ICP / NOT_ICP (score_icp, 568) ├── ofsted-inspection-summary/ # Reads report → CRM note, reader-aware │ │ ── UNATTENDED RUN · automation/ ── ├── run_daily_headless.ps1 # preflight → claude -p → sanity-check ├── build_digest_html.py # JSON → one persistent digest page (409) │ │ ── THE DATASET ── └── ofsted_reports_metadata.csv # 414 rows × 43 cols · every column an idempotency key