James StoneOPEN TO CONVERSATIONS
← All projectsGrowth / Marketing

Ofsted Signal Pipeline

Agentic pipeline · buying-intent signal

An unattended daily pipeline that turns the UK government's public Ofsted inspection feed into named, CRM-ready sales triggers — the day a provider is graded weak on exactly what Cognassist fixes.

Problem

In November 2025 Ofsted began publishing per-category inspection grades — including “Inclusion”, precisely what Cognassist sells into. Overnight the government started publishing a free, per-provider buying-intent signal, forever. But it ships as PDFs on a paginated site with no API: to use it you'd have to spot each new report, work out which legal entity it's about, pull three grades out of a PDF, match it to the CRM, qualify the provider, write a usable note, and judge whether it warrants a call — daily, across ~400 providers. A part-time job nobody had, and the signal decays to history in about six weeks.

Build

Built a fully unattended pipeline that runs once a day on one machine and writes live to Capsule CRM with no human in the loop. The design bet: split the work by kind — deterministic Python for anything where a wrong answer corrupts data (identity resolution, grade extraction, CRM matching, all writes), an LLM agent for the reading and judgement (qualification, note-writing, signal-flagging). The agent orchestrates the scripts; it never re-implements them. Every column of the single 43-field dataset doubles as an idempotency key, so any stage re-runs safely. ~4,100 lines of Python, four dependencies, one CSV — no database.

Outcome

Every morning it posts plain-English inspection summaries and outreach tasks straight into the CRM, and republishes one bookmarkable digest page. To date: 414 reports processed, UKPRNs resolved for 98.5%, 249 summary notes and 38 tasks written to Capsule. The commercial core is 38 named providers each carrying a public government document that says they're weak on exactly what Cognassist solves — already in the CRM, graded, with a task on someone's list.

Inside the repoRuns daily @ 07:00 · Python + Claude Code · no database
Ofsted-Reports/
├── CLAUDE.md                  # Agent context — read first
├── daily_pipeline_prompt.md   # The instructions the agent follows

│   ── DETERMINISTIC CORE · pipeline/ · 2,313 lines ──
├── ofsted_report_scraper.py   # Stage 1 · incremental scrape + PDF     (847)
├── enrich_ukprn.py            # Stage 2 · URN → UKPRN, authoritative    (222)
├── extract_grades.py          # Stage 3 · 3 toolkit grades from PDF     (243)
├── classify_capsule.py        # Stage 4 · match provider to CRM         (349)
├── add_capsule_prospects.py   # Stage 5b · create prospects     [LIVE]  (440)
├── create_capsule_tasks.py    # Stage 7 · signal → CRM task      [LIVE]  (262)

│   ── JUDGEMENT LAYER · .claude/skills/ ──
├── provider-icp-qualifier/    # Rubric: ICP / NOT_ICP     (score_icp, 568)
├── ofsted-inspection-summary/ # Reads report → CRM note, reader-aware

│   ── UNATTENDED RUN · automation/ ──
├── run_daily_headless.ps1     # preflight → claude -p → sanity-check
├── build_digest_html.py       # JSON → one persistent digest page       (409)

│   ── THE DATASET ──
└── ofsted_reports_metadata.csv  # 414 rows × 43 cols · every column an idempotency key