Research & Benchmarks

How we build open data infrastructure for the agent economy. Local LLMs, entity resolution, and classification at scale.

Over-combined, under-combined.
Thirteen businesses shared one identity because their domains had expired. Duke's lemur centre merged with its art museum. Half the pool had no handle at all. A day rebuilding entity resolution around one idea — a URL tells you who owns a page, not who a company is — with the traps, an honest grade, and the inference layer that settles what rules cannot.
The Internet Dial: What Is P(The Internet Is Fucked)?
This week two leaked PHP bugs and a CVSS 10.0 in GitLab had the NSA, FBI, CISA and the Prime Minister of Japan telling the world to consider taking its systems offline — and Wikipedia did. So we built the number: five measurable failure modes — mass exploitation, concentration, bots & slop, fragmentation, and the resilience counter-case — each scored from named indicators, pooled in log-odds, backcast to 2014, and re-run every Monday. Every indicator is at its all-time worst simultaneously; the recovery record is unbroken. Both facts are the model.
Vaccines Compared: Two AI Agents, One Question, Three Pages
Two AI agents researched the same vaccine question independently, froze their versions, then audited each other. The unified page: where 17 diseases land in both taxonomies, the product tier (IPV vs OPV, BCG, MMR, HPV), a one:many benefit map with economic and healthcare-demand nodes, a six-jurisdiction policy matrix — and an honest changelog of what each side got wrong, including a fabricated quote and a privacy failure the audit caught.
Vaccines Do Different Jobs — Instinct's Independent Version
The second agent's independent take, frozen September 30, 2026 before comparison: five overlapping jobs instead of four exclusive groups, product-level distinctions (IPV is not OPV), a benefits-and-risks framework that refuses one-score comparisons, and a six-jurisdiction policy analysis. Published as written — later corrections live on the unified page, never here.
The Four Endgames of Vaccination
My doctor buddy thinks every vaccine is the same public-health story. Five structural facts — person-to-person spread, human-only reservoir, transmission-blocking, target stability, carrier state — say otherwise: they sort 17 vaccine-preventable diseases into four groups with completely different endings. Winnable wars (measles, polio), forever shields (pertussis, HPV), damage control (flu, TB), and personal armor (tetanus — a vaccine with zero herd effect). With an interactive criteria graph, R0×VE scatter, and the mini-dataset behind every claim.
The Killer Part: Scoring 7 Pieces of the Unbundled GTM Stack
Teams are swapping $25k/mo data suites for Claude Code plus point APIs racing to $97/mo flat-rate. Which single component of that stack is worth building as a product? Seven candidates — entity resolution, waterfall enrichment, freshness monitoring, scored lists, agent-native access, verification, provenance — scored on six pinned, weighted dimensions. One wins by a distance, and a same-day audit of our own repos reached the same answer from the code side.
Two Assistants, One Workbench
Two AI assistants working for the same person started coordinating directly — and keeping a public log. Ideas, handoffs, shipped work, and the corrections: the first pull request one sent, the other caught a bug in before merge. An evidence-linked capability graph, a deliberately untested rivalry, and a hard rule: unmeasured means “not yet tracked”, never a made-up number. Updated as reviewed commits, not a mailbox mirror.
Smart If-Statements: We Put TypeSafe's Jev Through 2,000 Real Decisions
A new class of model claims frontier-level instinct at 40–1,000× cheaper than LLMs. We ran Jev against keyword rules, a logistic regression, llama3.1 and Claude Haiku on three real OnlyData classification tasks — measured accuracy, milliseconds and dollars. It beat Haiku on the 40-class task at a thirtieth of the cost, lost to a ten-second logistic regression everywhere we have labels, and failed our crawl-score task exactly the way its own limitations page predicts.
The Never-Ending Eval
Evals used to be a spreadsheet of prompts and right answers. Then agents happened. The unit of evaluation has moved four times in five years — completion, task, process, outcome — and most teams are still grading the first one. Step contracts, chain math, the three clocks ground truth arrives on, the prediction ledger, and why six structural fixes come before you ever touch the prompt.
The Doomsday Dial: What Is P(AI Kills Us All)?
An Anthropic researcher quit saying the labs are “gambling with our lives” — and Anthropic’s own alignment lead agreed the number is over 10%. So we built the number: a weighted log-odds pool of researcher surveys, superforecasters, prediction markets, frontier-lab insiders and the skeptical case, all converted to one 10-year horizon, backcast to 2014, and re-run every Monday. Every input cited, both sides represented, and the dial moves on the data.
Going Down the Sauna Rabbithole
How hot, how long, how cold? Benefit and risk curves you can drag, a temperature × time sweet-spot map, a sweat-loss experiment to run with your crew (weigh-in/weigh-out, the towel audit, the waterproof pan), and the physics of why a löyly steam burst stings at temps dry air shrugs off. Shout out The Rabbit Hole. ๐Ÿ‡
The Locked Tomb: A Reader's Crypt
A spoiler-gated companion to Gideon, Harrow, and Nona the Ninth. Tell it how far you've read and it unlocks exactly that much: full-spoiler summaries, a decoder ring for the confusing parts, the Nine Houses and their specialties, a ten-thousand-year timeline, character crossover — an interactive who-kills-whom graph where unread characters render as nameless ghosts, and a chapter-by-chapter map of Harrow with every card individually sealed.
The Economy on GLP-1s: When the Consumption Engine Loses Its Appetite
68% of US GDP is us buying stuff — and GLP-1 drugs are the first technology that shrinks appetite at population scale. Grocery baskets, bariatric surgery, insulin, and alcohol are already denting; the price dam ($1,349 → $15 generics abroad) is breaking. Sourced evidence, winners and losers, and an interactive 2026–2035 GDP model with sliders. ๐ŸŽง With audio edition.
Small-Scale Magic: a Mac Studio, a Queue, and a Production App
CaChink users name a song and watch an agent build them a playable tab, live — solos complete, original rig documented, every note validated before deploy. The trick: a hardened app on traditional infra, one queue row of coupling, and all the intelligence on a Mac Studio at ~$0 — swappable for API inference the day it matters. A pattern already powering four apps.
The Personal Resource Graph
40+ scheduled jobs, 315 tables, two chat listeners, a fleet of local models — and silent rot. One typed graph of everything that runs, with freshness contracts, a model registry, and enforcement that blocks commits. In its first hour it caught a job that had been failing every 15 minutes for days.
The Iceberg: How to Make AI Tools That Actually Work
The demo is a Telegram message; the product is everything under the waterline. Why almost every personal AI tool dies as a demo, the layered architecture (context repo → dispatch → protocols → cron → propagation rules) that made ours actually change daily life — and a public starter repo to clone.
The token-maxing canyon.
Subscription buffets trained a generation of token-maxers. The buffet is closing โ€” Claude Code is out of the $20 plan, weekly caps are tightening, and rolling windows are shrinking. Here's the math behind why it had to, and the incentive flip we should actually want.
The night we wrote our data's axioms.
From a 'Not found' page to 9 of 9 hard data invariants enforced. A six-hour autopilot cleanup that collapsed 411 duplicate company rows, canonicalized 1,042 ugly slugs, and backfilled 21,968 missing profiles โ€” and the daily job that keeps it that way.
OnlyData already labeled 90 companies AI-native. The AR scorer ignored them.
Of 90 companies the ai_native_v1 classifier tagged as genuinely AI-native, only ONE scored above AR 50. Character.AI scored 0. Cognition/Devin scored 11. The fix isn't a learned model โ€” it's a one-line floor.
The Shape of AI: How Similarity Scores Reveal Hidden Patterns
We embedded 24K companies into 384-dimensional vector space and explored what the geometry tells us about industry boundaries, agent readiness, and invisible connections. Interactive D3 scatters, spider charts, and force-directed graphs.
Fixing the Vibe Coder Stack: We Asked Claude to Audit Our Dataset Through Our Own MCP
Our auto-classified Vibe Coder Stack had 94 cos in 38 categories — including a gas station and an optometrist. We pointed Claude at it through the OnlyData MCP, got a brutal audit, and rebuilt it as a 90-company curated list across 9 sublists.
Eating Our Own Dogfood: How We Used Our Own MCP to Improve Our Data
We shipped the OnlyData MCP, then pointed it at our own catalog and found a massive blind spot in the agentic AI layer. 199 companies promoted from a private list to a public dataset — 155+ of them net new.
Agent Readiness: The Attribute Nobody Has
We scanned 8,250+ business websites for AI agent readiness with Algorithm E (Spread, v3). Six categories, subdomain probing, 0-100 score. Average 24. Only 9.3% grade A. Just 3% cross the well-behaved-SaaS ceiling of 50. Digits tops the list at 93.
The Prompt That Changed Everything: B2B Classification from 17% to 77%
We tested 5 local LLMs with 3 prompt strategies on 30 real Boise businesses. Chain-of-thought reasoning took accuracy from 17% to 77%. gemma4:e2b hits 73% on name alone.
Small Models, Big Questions: The Real Bottleneck Isn't the Model
5 models, 5 rounds, 30 real businesses. We went from 0% NAICS accuracy to 83% by splitting semantics from codes. The taxonomy was the bottleneck, not the model.