Research & Benchmarks

How we build open data infrastructure for the agent economy. Local LLMs, entity resolution, and classification at scale.

Smart If-Statements: We Put TypeSafe's Jev Through 2,000 Real Decisions
A new class of model claims frontier-level instinct at 40–1,000× cheaper than LLMs. We ran Jev against keyword rules, a logistic regression, llama3.1 and Claude Haiku on three real OnlyData classification tasks — measured accuracy, milliseconds and dollars. It beat Haiku on the 40-class task at a thirtieth of the cost, lost to a ten-second logistic regression everywhere we have labels, and failed our crawl-score task exactly the way its own limitations page predicts.
The Never-Ending Eval
Evals used to be a spreadsheet of prompts and right answers. Then agents happened. The unit of evaluation has moved four times in five years — completion, task, process, outcome — and most teams are still grading the first one. Step contracts, chain math, the three clocks ground truth arrives on, the prediction ledger, and why six structural fixes come before you ever touch the prompt.
The Doomsday Dial: What Is P(AI Kills Us All)?
An Anthropic researcher quit saying the labs are “gambling with our lives” — and Anthropic’s own alignment lead agreed the number is over 10%. So we built the number: a weighted log-odds pool of researcher surveys, superforecasters, prediction markets, frontier-lab insiders and the skeptical case, all converted to one 10-year horizon, backcast to 2014, and re-run every Monday. Every input cited, both sides represented, and the dial moves on the data.
Going Down the Sauna Rabbithole
How hot, how long, how cold? Benefit and risk curves you can drag, a temperature × time sweet-spot map, a sweat-loss experiment to run with your crew (weigh-in/weigh-out, the towel audit, the waterproof pan), and the physics of why a löyly steam burst stings at temps dry air shrugs off. Shout out The Rabbit Hole. ๐Ÿ‡
The Locked Tomb: A Reader's Crypt
A spoiler-gated companion to Gideon, Harrow, and Nona the Ninth. Tell it how far you've read and it unlocks exactly that much: full-spoiler summaries, a decoder ring for the confusing parts, the Nine Houses and their specialties, a ten-thousand-year timeline, character crossover — an interactive who-kills-whom graph where unread characters render as nameless ghosts, and a chapter-by-chapter map of Harrow with every card individually sealed.
The Economy on GLP-1s: When the Consumption Engine Loses Its Appetite
68% of US GDP is us buying stuff — and GLP-1 drugs are the first technology that shrinks appetite at population scale. Grocery baskets, bariatric surgery, insulin, and alcohol are already denting; the price dam ($1,349 → $15 generics abroad) is breaking. Sourced evidence, winners and losers, and an interactive 2026–2035 GDP model with sliders. ๐ŸŽง With audio edition.
Small-Scale Magic: a Mac Studio, a Queue, and a Production App
CaChink users name a song and watch an agent build them a playable tab, live — solos complete, original rig documented, every note validated before deploy. The trick: a hardened app on traditional infra, one queue row of coupling, and all the intelligence on a Mac Studio at ~$0 — swappable for API inference the day it matters. A pattern already powering four apps.
The Personal Resource Graph
40+ scheduled jobs, 315 tables, two chat listeners, a fleet of local models — and silent rot. One typed graph of everything that runs, with freshness contracts, a model registry, and enforcement that blocks commits. In its first hour it caught a job that had been failing every 15 minutes for days.
The Iceberg: How to Make AI Tools That Actually Work
The demo is a Telegram message; the product is everything under the waterline. Why almost every personal AI tool dies as a demo, the layered architecture (context repo → dispatch → protocols → cron → propagation rules) that made ours actually change daily life — and a public starter repo to clone.
The token-maxing canyon.
Subscription buffets trained a generation of token-maxers. The buffet is closing โ€” Claude Code is out of the $20 plan, weekly caps are tightening, and rolling windows are shrinking. Here's the math behind why it had to, and the incentive flip we should actually want.
The night we wrote our data's axioms.
From a 'Not found' page to 9 of 9 hard data invariants enforced. A six-hour autopilot cleanup that collapsed 411 duplicate company rows, canonicalized 1,042 ugly slugs, and backfilled 21,968 missing profiles โ€” and the daily job that keeps it that way.
OnlyData already labeled 90 companies AI-native. The AR scorer ignored them.
Of 90 companies the ai_native_v1 classifier tagged as genuinely AI-native, only ONE scored above AR 50. Character.AI scored 0. Cognition/Devin scored 11. The fix isn't a learned model โ€” it's a one-line floor.
The Shape of AI: How Similarity Scores Reveal Hidden Patterns
We embedded 24K companies into 384-dimensional vector space and explored what the geometry tells us about industry boundaries, agent readiness, and invisible connections. Interactive D3 scatters, spider charts, and force-directed graphs.
Fixing the Vibe Coder Stack: We Asked Claude to Audit Our Dataset Through Our Own MCP
Our auto-classified Vibe Coder Stack had 94 cos in 38 categories — including a gas station and an optometrist. We pointed Claude at it through the OnlyData MCP, got a brutal audit, and rebuilt it as a 90-company curated list across 9 sublists.
Eating Our Own Dogfood: How We Used Our Own MCP to Improve Our Data
We shipped the OnlyData MCP, then pointed it at our own catalog and found a massive blind spot in the agentic AI layer. 199 companies promoted from a private list to a public dataset — 155+ of them net new.
Agent Readiness: The Attribute Nobody Has
We scanned 8,250+ business websites for AI agent readiness with Algorithm E (Spread, v3). Six categories, subdomain probing, 0-100 score. Average 24. Only 9.3% grade A. Just 3% cross the well-behaved-SaaS ceiling of 50. Digits tops the list at 93.
The Prompt That Changed Everything: B2B Classification from 17% to 77%
We tested 5 local LLMs with 3 prompt strategies on 30 real Boise businesses. Chain-of-thought reasoning took accuracy from 17% to 77%. gemma4:e2b hits 73% on name alone.
Small Models, Big Questions: The Real Bottleneck Isn't the Model
5 models, 5 rounds, 30 real businesses. We went from 0% NAICS accuracy to 83% by splitting semantics from codes. The taxonomy was the bottleneck, not the model.