Data · Jev's first week · 16 to 23 September 2026
A week of Jev, sorted
I sorted 5,950 posts from Jev's first week, read the usage the gateways show, and counted the rivals. This is what the data says. What I make of it is a separate piece.
Jev is TypeSafe AI's decision model. It doesn't write text. You give it a state and a typed question, pick one of these, score this, yes or no, and it answers in a few hundred milliseconds with a probability. TypeSafe's launch claim was 20 to 200x faster and 40 to 400x cheaper than an LLM for that kind of question; "100x cheaper, 10x faster" is the shorthand the week settled on.
I built Pong on it the week it came out. The post did well, and nearly every comment asked the same thing: fine, you built a game, what's it for? I tried to answer and couldn't. Every use case I reached for was either already solved or harder than a cheaper yes-or-no. So I stopped guessing and measured: every post in OpenChamber's Jev feed for the first week, sorted and audited; the two gateways that publish usage; and the rivals that turned up. The method and the audit are at the end.
The short version
- Jev is being called at scale, and the money is small. On 23 September it was a quarter of all requests on Vercel's AI Gateway and 2% of its tokens; on OpenRouter it served more requests that week than GPT-5.6 Luna, for an eighth of the spend. TypeSafe publishes no usage number, and 96% of OpenRouter's Jev requests come from apps that don't say who they are.
- The buzz is about using it, not about what it makes possible. Four in five posts compared Jev with nothing, and none of the 91 builds most likely to show something new did something that was unavailable before.
- TypeSafe announced 40 to 400x cheaper than frontier models, and against frontier models it holds. The posts claimed 28x. Measured against the small models you'd actually use, the median is 7.3x on cost and 4.7x on latency, often with better accuracy.
- Twenty rivals in a week, mostly built by one person in days on open weights. None matches Jev's mix yet, and no big provider has shipped one.
1 · How big it really is
A quarter of Vercel's gateway requests, 2% of its tokens, and nobody can say who's behind it
TypeSafe hasn't published a usage number, so the gateways are the only view anyone has, and they show what the posts can't: Jev is being called at a scale that dwarfs the demos. What the chart can't show is who's calling it. Five named apps (a data-labeling app with 12.9 million requests, Magnific, mirasim, the Asian Development Bank's evidence portal, a Naver rubric judge) account for 4% of OpenRouter's Jev requests. The rest is anonymous. Demand outran TypeSafe: 140,000 people were let off the waitlist in 36 hours, signups were paused a week in, and the API had four short outages.
Show the numbers
| Day | OpenRouter requests | Prompt tokens | Billed | Vercel: share of requests | Vercel: share of tokens |
|---|---|---|---|---|---|
| 17 Sep | 5.9% | 0.34% | |||
| 18 Sep | 28,243,338 | 73B | $3,069 | 13.6% | 0.77% |
| 19 Sep | 52,641,640 | 140B | $5,852 | 26.3% | 1.79% |
| 20 Sep | 64,434,110 | 196B | $8,169 | 20.8% | 2.00% |
| 21 Sep | 82,379,677 | 256B | $10,723 | 21.2% | 2.04% |
| 22 Sep | 83,638,877 | 246B | $10,295 | 22.0% | 1.75% |
| 23 Sep | 125,647,722 | 327B | $13,655 | 25.2% | 1.81% |
Who is trying it
| Signal | Number | When | Source |
|---|---|---|---|
| People let off the waitlist | 140,000 in under 36 hours | 17 Sep | Diogo Almeida, TypeSafe |
| Waitlist removed | "Jev is now available to everyone." | 20 Sep | TypeSafe |
| Signups paused | "an immense swell of demand"; not reopened by 24 Sep | 22 Sep | TypeSafe |
| Vercel AI Gateway, day one | nearly 13% of paid teams by hour 24; every other recent launch under 7% after a day | 18 Sep | Vercel |
| Vercel AI Gateway, latest day | used by 37.8% of teams, first; the primary model by tokens for 33.8% of teams | read 24 Sep | Vercel leaderboards |
| SDK downloads, launch week | 298,412 JavaScript, 46,177 Vercel provider, 331,517 Python (automated installs included) | read 24 Sep | npm, pypistats |
| New GitHub repositories matching "jev" | 8,106 since 15 Sep, against 56 the week before | read 24 Sep | GitHub search |
| Discord members | 107,727 | read 24 Sep | Discord |
| Named apps on OpenRouter | five apps hold 4.3% of Jev requests; a data-labeling app alone 12.9 million; the rest is anonymous | read 24 Sep | OpenRouter |
| Incidents | API down 5, 18, 2 and 12 minutes on 17, 20, 21 and 23 Sep; none in July or August | read 24 Sep | TypeSafe status page |
The calls are real and tiny. A request isn't a decision, a gateway can't tell trying from using, and Jev was free on Vercel all week, so read the Vercel line as a ceiling.
2 · Weighted by money
More requests than GPT-5.6 Luna, for an eighth of the money
This is what 100x cheaper looks like on a bill. Priced on the same day's tokens, Jev sits with the cheapest small models, and one of them, DeepSeek V4 Flash, is cheaper per token. The 244x is only there if you assume the calls would have gone to a frontier model, and nobody sends yes-or-no questions to Fable. Most of these calls exist because they cost almost nothing, not because they replaced something expensive.
Show the numbers
| Model | Prompt price, $ per million tokens | Output price | Cost of 23 Sep's Jev tokens | Multiple of Jev's bill |
|---|---|---|---|---|
| DeepSeek V4 Flash 0731 | 0.030 | 0.32 | $10,209 | 0.7x |
| Jev | 0.042 | 0.00 | $13,655 | billed |
| GPT-5 Nano | 0.050 | 0.40 | $16,848 | 1.2x |
| Ministral 3B | 0.100 | 0.10 | $32,816 | 2.4x |
| Gemini 3.5 Flash-Lite | 0.300 | 2.50 | $101,211 | 7.4x |
| Claude Haiku 4.5 | 1.000 | 5.00 | $333,183 | 24x |
| GPT-5.6 Sol | 2.000 | 10.00 | $666,365 | 49x |
| Claude Fable 5.1 | 10.000 | 50.00 | $3,331,826 | 244x |
A price comparison, not a quality one: nobody has run these tasks on the other models.
3 · What people built
We can't see what everyone runs. We can see what they show.
The gateways show volume with no names. The posts show names with no volume. So the rest of this page is about the 5,595 posts: what people said Jev decides, what they measured, and what they compared it with. Two things stood out. Games are the biggest single family, not classification. And a fifth of the posts never say what the decision is at all.
Show the numbers
| Group | Posts | Share | No measurement | Measured demo | Labeled production, not confirmed | Production, confirmed | In a live loop | Needs under 300 ms |
|---|---|---|---|---|---|---|---|---|
| Decisions about data (classify, route, extract, rerank, judge, telemetry) | 1,936 | 35% | 1,135 | 778 | 19 | 4 | 1% | 0% |
| Decisions inside games and control loops (games, simulations, robots) | 1,071 | 19% | 791 | 280 | 0 | 0 | 72% | 66% |
| Decisions inside agent loops (gate a tool call, pick the next browser action, prune context) | 603 | 11% | 394 | 207 | 1 | 1 | 3% | 1% |
| Decisions about what a person just said or typed (moderation, voice, live chat, collaboration) | 463 | 8% | 338 | 121 | 4 | 0 | 45% | 18% |
| Decisions about money (trading bots) | 268 | 5% | 145 | 122 | 1 | 0 | 39% | 2% |
| No decision stated (meta, benchmarks of the model, wrappers) | 1,254 | 22% | 1,032 | 215 | 4 | 3 | 1% | 0% |
| All posts | 5,595 | 100% | 3,835 | 1,723 | 29 | 8 |
Read everything below knowing the week was mostly demos: two thirds of posts measured nothing, and a third measured a demo. Production numbers in posts are rare (8 of 5,595 held up on the audit), which fits the gateway picture: the volume sits with apps that don't post.
4 · Was any of it new?
The buzz is about using it, not about what it makes possible
Four in five posts compared Jev with nothing. That matters because Jev is doing jobs that already had tools. If it were solving something nobody could solve before, there'd be nothing to compare with. It isn't, so the missing baseline is the tell.
Show the numbers
| Baseline | Posts | Share of posts | Share of views | Audit estimate (95% interval) |
|---|---|---|---|---|
| Nothing | 4,498 | 80.4% | 74.9% | 80% (77 to 82) |
| Frontier LLM | 630 | 11.3% | 14.3% | 11.3% (10 to 13) |
| Small LLM | 239 | 4.3% | 6.9% | 5.0% (4 to 7) |
| Rules or regex | 138 | 2.5% | 2.2% | |
| Classic classifier or ML | 59 | 1.1% | 0.5% | |
| Vendor API | 31 | 0.6% | 1.3% | |
| The tools Jev would replace (the last three) | 228 | 4.1% | 3.7% (3 to 5) |
The audit went further. It took the 91 posts where the first-pass labels found a measured decision inside a live system with a person waiting on it, the builds most likely to need both the speed and the price, and asked what a team would have used before Jev. An LLM for 64. Rules for 12. A vendor API for 11. A classic model for 4. None did something that was unavailable before. (85 distinct builds; 38 still meet the test on the auditor's own labels.)
Show the numbers
| What would have done the job before | Posts | Share of the candidates |
|---|---|---|
| An LLM | 64 | 70% |
| Rules or heuristics | 12 | 13% |
| A vendor API | 11 | 12% |
| A classic model | 4 | 4% |
| Unavailable at any price | 0 | 0% |
| Distinct builds among them | 85 | |
| Job: feed or comment filters | 22 | 24% |
| Job: typing or form fill | 13 | 14% |
| Job: voice commands | 13 | 14% |
| Job: triage | 6 | 7% |
| Job: voice turn-taking | 6 | 7% |
| Job: guards | 5 | 5% |
| Job: speech scoring | 5 | 5% |
| Job: trading decisions | 5 | 5% |
| Job: classification | 3 | 3% |
| Job: computer use | 3 | 3% |
| Job: games or simulations | 3 | 3% |
| Job: alerts | 2 | 2% |
| Job: live control | 2 | 2% |
| Job: search | 2 | 2% |
| Job: personal tools | 1 | 1% |
Faster and cheaper versions of jobs that had a tool. A real change to who can try them; one week in, not a change to what gets built.
5 · Announced, claimed, measured
TypeSafe's 40 to 400x is against frontier models, and holds there. Against the small models you'd actually use, the median is 7.3x.
The launch claim was against frontier models, and on the four published frontier comparisons it holds. The posts repeated it. Where someone put Jev against a small model on the same task and published the numbers (I found 12 such comparisons, including my own Pong runs) the gap shrank to a median of 7.3x on cost and 4.7x on latency (6x and 5.3x without my own runs), with Jev at or above the small model on accuracy for most bounded questions. Against frontier models the multiples are real: 97x to 580x on cost.
Show the numbers
| Who | The task | Against | Cheaper by | Faster by | Accuracy | Source |
|---|---|---|---|---|---|---|
| Jev Pong (me) | the next paddle move, 30 states | Ministral 3B (small) | 2.8x | 1.8x | Jev 28 of 29, Ministral 26 of 30 | link |
| Jev Pong (me) | the next paddle move, 30 states | GPT-5.4 Nano (small) | 10x | 3.6x | Jev 28 of 29, Nano 29 of 30 | link |
| Jev Pong (me) | the next paddle move, 30 states | Claude Haiku 4.5 (small) | 44x | 3.6x | Jev 28 of 29, Haiku 30 of 30 | link |
| Jon Reed, Near Here | reject unsuitable event listings, 50 cases | Mistral Small 4 (small) | 8.6x | 4.9x | Jev 96%, Mistral 84% | link |
| Jon Reed, Near Here | reject unsuitable event listings, 50 cases | Gemini 3.5 Flash-Lite (small) | 58x | 5.8x | Jev 96%, Gemini 86% | link |
| Emil Lindfors | 11 typed questions over 24 hearing responses | DeepSeek V4.1 Flash (small) | 6x | 8.4x | stance 20 of 24 each; substance Jev 19, DeepSeek 14 | link |
| Dan Willoughby, Sniff Test | ten boolean questions a paragraph, 54 clean paragraphs | Claude Haiku 4.5 (small) | 33x | 11x | false flags: Jev 1, Haiku 37 | link |
| Jalil Laaraichi, OpenWork | page, ticket or ignore over 8,000 log lines | GPT-5.6 Luna (small) | 5.2x | not published | Jev 100% recall and 0 false pages, Luna 96.2% | link |
| CodeAlive | block or allow agent input, 58 messages | gpt-oss-120b (small) | 4x | 4.7x | Jev 9 of 9 hostile blocked and 0 of 49 real blocked; gpt-oss 8 or 9 of 9 | link |
| anessbelbati, rerank bench | search reranking across 14 datasets | Cohere Rerank 4 Pro (small) | 5.6x | 2x | nDCG@10 0.692 against 0.691 | link |
| Taishi Morinaga, Classmethod | four-tier model routing, 40 calls | Gemini 3.5 Flash (small) | not published | 3.2x | Jev 10 of 10 on every pattern | link |
| anpicasso, Hermes approvals | approve, deny or escalate 153 real commands | a small chat model (small) | not published | 9.8x | 10 escalations to a human instead of 42 | link |
| LangWatch | judge 300 support conversations against human labels | Claude Opus 5 (frontier) | 97x | 8.2x | Jev 97% agreement, Opus 91% | link |
| Dan Shipper, Every | four writing checks over 12 passages | Claude Fable 5.1 (frontier) | 580x | 25x | Jev 6 of 7 planted defects, Fable 7 of 7 | link |
| awlevin, computer use | the next action from an OCR of the screen | Claude Opus 5 (frontier) | 160x | 14x | not measured | link |
| Dan Willoughby, Sniff Test | ten boolean questions a paragraph, 54 clean paragraphs | Claude Opus 5 (frontier) | 239x | 36x | false flags: Jev 1, Opus 0 | link |
The cost uses people measured
| Job | Who | What they measured | Source |
|---|---|---|---|
| Evals and judging | LangWatch | 97% agreement with human labels over 300 support conversations; 10,000 conversations judged for $0.32 in 73 seconds, against $31 and ten minutes on Opus 5 | langwatch.ai |
| Evals and judging | Mike Taylor and Dan Shipper, Every | 777 judgments in under 0.7 seconds; 0.35 s a passage against 8.8 s on Fable 5.1, about 580 times cheaper | every.to |
| Classification | Hassan, 1kpapers | 1,018 papers sorted into 24 topics for $0.08, 256 ms a paper | X |
| Classification | Emil Lindfors | $0.22 per 1,000 documents; 0.32 s a document | lindfors.no |
| Triage | Jalil Laaraichi, OpenWork | log triage with 0 false pages; $0.062 against $0.32 per 3,000 logs | dev.to |
| Security screening | Gaurav Gosain | 96.5% on 662 blind prompt-injection and vulnerable-code cases; p50 325 ms | GitHub |
| Agent approvals | anpicasso, Hermes | 153 real commands: 10 escalations to a human instead of 42; 405 ms against 3,968 ms | GitHub |
| Computer use | awlevin | next action from an OCR of the screen: $0.0002 a step against $0.032; 0.13 to 0.38 s against 5.2 s | GitHub |
| Browser agents | Gregor Zunic, Browser Use | a real flight search 25% faster end to end; 101 protocol calls instead of 1,092 | GitHub |
| Context pruning | tamara | score every tool call and result, drop the stale ones: 1M tokens to 86K | madewithjev |
| Reranking | anessbelbati | parity with Cohere Rerank 4 Pro on nDCG at half the latency | GitHub |
The honest number for a build-or-buy is single digits against a small model. Worth having, and it moves the sweet spot from ten times the decisions at a tenth of the bill to a few times the decisions at a fraction of it. The jobs are evals, triage, extraction, reranking and agent gating, and small-model prices have been falling roughly 10x a year on their own.
6 · Who needs it fast
Outside games, speed is a handful of demos, and they share one shape
Put what people built against how soon the decision has to land and there is one hot spot. Games. Everything else that needs an answer in under 300 ms fits in two small cells: typing and voice turn-taking. The volume sits with data decisions that can wait seconds, and with posts that never say what the decision is.
Show the numbers
| Group | Under 300 ms | Under a second | Seconds or slower | Unclear | All posts |
|---|---|---|---|---|---|
| Decisions about data | 4 (0%) | 540 (28%) | 1,239 (64%) | 153 (8%) | 1,936 |
| Decisions inside games and control loops | 707 (66%) | 230 (21%) | 74 (7%) | 60 (6%) | 1,071 |
| Decisions inside agent loops | 5 (1%) | 39 (6%) | 557 (92%) | 2 (0%) | 603 |
| Decisions about what a person just said or typed | 84 (18%) | 255 (55%) | 99 (21%) | 25 (5%) | 463 |
| Decisions about money | 6 (2%) | 44 (16%) | 194 (72%) | 24 (9%) | 268 |
| No decision stated | 3 (0%) | 146 (12%) | 71 (6%) | 1,034 (82%) | 1,254 |
| All groups | 809 (14.5%) | 1,254 (22.4%) | 2,234 (39.9%) | 1,298 (23.2%) | 5,595 |
The audit confirms the shape: a tenth of posts need a decision in under 300 ms, and nine in ten of those are games. Voice, live chat and collaboration, where a person is waiting, are 4% (3 to 7) of posts, and none is measured in production. The few live builds that exist mostly sit in one repo, Nader Dabit's jev-experiments, and they share a shape: several of your own questions on every event, inside the turn, from a model you didn't train. That's the one pattern nothing served before for anyone without an ML team.
Show the numbers
| Audited estimate | Estimate | 95% interval | From |
|---|---|---|---|
| Posts that need a decision in under 300 ms | 11.0% | 9.5% to 12.5% | 76 of 150 re-read posts still under 300 ms, plus 91 that only Opus puts there |
| The same, on the audit's samples alone | 10.3% | 8.2% to 14.3% | neither model's word |
| Games, among those posts | 90.8% | 82.2% to 95.5% | 69 of 76 |
Show the numbers
| Family | Labeled production | Measured demo | Demo, no numbers | Proposal or commentary | Flagged realtime |
|---|---|---|---|---|---|
| Trading and markets | 1 | 60 | 43 | 0 | 104 |
| Voice and turn-taking | 0 | 21 | 48 | 0 | 69 |
| Live chat and streams | 1 | 9 | 54 | 0 | 64 |
| Collaboration and typing | 0 | 11 | 51 | 0 | 62 |
| Browser and computer use | 0 | 3 | 16 | 0 | 19 |
| Moderation and guardrails | 0 | 9 | 5 | 1 | 15 |
| Other or meta | 0 | 1 | 14 | 0 | 15 |
| Classification and routing | 0 | 4 | 5 | 0 | 9 |
| Data and telemetry | 0 | 0 | 7 | 0 | 7 |
| Search, rerank, extraction | 0 | 3 | 3 | 0 | 6 |
| Evals and judging | 0 | 2 | 2 | 0 | 4 |
| Agent harness and gating | 0 | 0 | 2 | 0 | 2 |
| All outside games | 2 | 123 | 250 | 1 | 376 |
The live builds, by name
| What | Who | What they measured | Source |
|---|---|---|---|
| Voice turn-end on every partial transcript | Nader Dabit | 108 ms round trip; the assistant replies 351 ms after the speaker stops instead of 1,000 ms | GitHub |
| Hold every chat message before it renders | Nader Dabit | 45 messages a second held 124 ms each; 381 of 383 harmful messages caught, 0 of 958 clean ones blocked, on a labeled set | GitHub |
| Judge a draft between keystrokes | Nader Dabit | 105 ms end to end, 85 judgments a second while typing; 4 of 4 risky drafts caught, 0 false blocks | GitHub |
| Nine questions per customer message, eight chats at once | Nader Dabit | panel refresh 93 ms | GitHub |
| Live minutes that flag a reversed decision | Nader Dabit | 131 ms from end of utterance to screen; action items 96% recall, 92% precision | GitHub |
| A 300-message-a-second stream, every message judged | Nader Dabit | 124 ms end to end, zero backlog, $37.86 an hour | GitHub |
| Turn detection for voice agents | Kwindla Hultman Kramer | 92.6% at 296 ms, against an LLM at 81.3% and 1,008 ms | X |
| Voice and gesture on a canvas, eight questions per partial transcript | gaborishka | 300 to 550 ms a decision | GitHub |
| Per-turn decisions as a LiveKit plugin | sidxh | a proposal; 0 comments, 0 reactions | GitHub |
| The same shape, local, on a laptop | mizorewww, laya-mlx | 7 to 14 ms a decision on an M3 Max; no cloud in the path | GitHub |
Where speed and cost compound
The use cases people talked about, what they were done with before, and what a general decision model changes. Every one is a demo or a proposal.
| Use case | How it was done before | What a decision model changes | Speed, cost, or both | Evidence so far |
|---|---|---|---|---|
| A voice control plane: speak now, is this a command, escalate | silence timers; a trained classifier per product; or an LLM at about a second | several product-specific questions on every partial transcript, about 300 ms, no training | speed | two head-to-heads (92.6% at 296 ms against 81.3% at 1,008 ms); demos |
| A moderation policy per room, applied before fan-out | keyword filters; vendor taxonomies; an LLM per message at 0.5 to 3 s, so after the fact | a prose policy the room owner writes, several hazards per message, held about 120 ms | both | a synthetic set: 99.5% caught, 0 clean messages blocked |
| Who is this for? Gating an assistant in a group | wake words, @-mentions, or an LLM at about a second | a decision on every utterance at about 340 ms | speed | demos, no accuracy numbers |
| A human-takeover trigger on every turn | a trained escalation classifier, or an LLM judge adding 1 to 3 s | left on for every turn, before the reply streams | both | 10 escalations instead of 42, 405 ms against 3,968 ms; one measured demo |
| A live audience board, every message scored | sampled or batch sentiment; an LLM per message at $140 to $950 an hour | every message, categories redefined as you go, $37.86 an hour at 300 messages a second | cost | a measured demo |
| Live minutes that flag a reversed decision | a summary after the meeting | flagged in about 130 ms, while everyone is still in the room | cost, mostly | one repo, on replay |
The latency story belongs to games and to local models: a local Laya answers in 7 to 14 ms on a laptop. The live human tier is where cost and speed compound into something new, and after a week it holds demos.
7 · Who else is coming
Twenty rivals in a week, most built by one person in days
The moat is smaller than the launch suggested. SemIf was up within a day. AutoJev was trained by agents in 20 hours on one H200 for $3,100. Reflex was one Shopify engineer over three days. JevK5, a week old, is second on JevBench. Each rival moves one or two knobs (15 of the other top-20 systems beat Jev on cost, 8 on speed, 1 on calibration, none on intelligence), and none matches Jev's mix yet. No big provider has announced a decision endpoint. And people built local copies partly because they couldn't get in.
Show the numbers
| Name | Who | First seen | Type | Competes on | Independent check | Source |
|---|---|---|---|---|---|---|
| SemIf (was OpenJev) | TheoLeeCJ | 16 Sep | open-weights model | run yourself, cost, accuracy | JevBench #9 (was #2); 4,190 stars | link |
| jevlike | vinnylarouge | 16 Sep | open-weights model | run yourself, cost, latency | none found | link |
| decider-2b | Mapika | 16 Sep | open-weights model, 1.9B | run yourself, cost, latency | S1Bench: behind SimpleJev and Reflex among free options | link |
| djev (DiffusionGemma as Jev) | Maisa | 19 Sep | open-weights model and hosted endpoint | latency, cost, generality, run yourself | JevBench #6 | link |
| openjev-sglang | Eric Zhang | 17 Sep | hosted endpoint | latency, run yourself | none found | link |
| Qwen 3.8 27B on Cerebras | Shannon (@iamMrDuncan) | 17 Sep | hosted model | latency, accuracy | none found | link |
| Kev | Jared Palmer | 17 Sep | open-weights models, 0.5B to 9B | run yourself, cost, accuracy | JevBench #24 (4B), #45 (0.6B); 6,598 stars | link |
| Reflex | Kshetrajna Raghavan (Shopify) | 17 Sep | open-weights model, in the browser | run yourself, cost | JevBench #5; cheaper than Jev | link |
| SimpleJev | Eugene Cheah (Featherless AI) | 18 Sep | library and hosted endpoint, any model, vision | generality, cost, run yourself | best free option on S1Bench (19 Sep) | link |
| Laya | Convai Innovations | 18 Sep | open-weights model, 421M | latency, cost, run yourself | JevBench #36; near chance zero-shot; a base to specialize | link |
| localjev | GitHub Next | 18 Sep | local port of Jev's API | run yourself | none found | link |
| Verdict 2.0 | Hemant (@heman10x) | 18 Sep | open-weights model, 151M, in the browser | accuracy, cost, run yourself | JevBench #54 | link |
| DeepSeek V4.1 Flash behind a Jev-style endpoint | Nick Khami | 19 Sep | hosted endpoint | generality, run yourself | none found | link |
| Distilled 4B | Taro L. Saito | 19 Sep | fine-tune recipe | latency, accuracy, run yourself | none found | link |
| Jeff | Logan Markewich | 19 Sep | self-hosted drop-in | run yourself, cost | JevBench #35 | link |
| laya-mlx | mizorewww | 19 Sep | local port, Apple silicon | latency, run yourself | 7 to 14 ms on an M3 Max; 6,150 stars | link |
| AutoJev | Denis Yarats | 19 Sep | open-weights model, trained by agents | run yourself, cost | none found; 20 hours on one H200, $3,100 (the author) | link |
| Winnow-12B | Eldan Ring | 20 Sep | open-weights vision model | generality, run yourself | JevBench #4 | link |
| Hopper | HopitAI | 21 Sep | open-weights model | latency, cost, calibration, run yourself | JevBench #3; better calibrated than Jev | link |
| JevK5 | allebee | 22 Sep | open-weights model, 4.2B | latency, cost, run yourself | JevBench #2, 62.0 against Jev 63.3 | link |
| JevBench (benchmark) | Florian S, Benchmark Heaven | 19 Sep | benchmark | the scoreboard | 77 systems ranked on 23 Sep | link |
| S1Bench (benchmark) | Cuth (@ItsCuthulhu) | 19 Sep | benchmark | the scoreboard | 40 alternatives on 20 Sep | link |
What it took to get close: one person, a few days, open weights, and at most a few thousand dollars.
8 · Where it is
What the week adds up to
- Jev is being called at scale, mostly by apps that don't say who they are, and the money is small.
- Against comparable small models the gap is single digits, not 100x and 10x.
- The uses that can be named are decisions about data: labeling, routing, judging, extraction. Possible before, cheaper now, and getting cheaper regardless.
- Speed and cost compound in one place, decisions about a person's words in real time. It's 8% of posts, a few demos, and nothing measured in production.
- None of the 91 builds most likely to show something new did something that was unavailable before.
- Twenty rivals in a week, built cheaply, none matching the mix yet, and no big provider.
What to watch
- 2 October. Jev stops being free on Vercel on 25 September. If its share of requests is still around a quarter a week later, the volume was use; if it halves, a lot of it was free-tier tinkering. I'll re-read the same export, OpenRouter's daily curve, and whether the five named apps are still there.
- When TypeSafe reopens signups, and whether it publishes a usage number.
- The first decision endpoint from a big provider.
- The first production number in the live tier: a voice agent, a chat room, a takeover trigger, with a volume attached.
What I think it means is a separate piece. This page is the data.
Method
How this was measured
The posts come from OpenChamber's Jev feed, snapshot 23 September 18:47 UTC: 5,950 posts from 4,593 authors between 16 and 23 September, as OpenChamber selected them. Without 7 duplicates, the 347 posts that don't use Jev and one post the labeling model refused, 5,595 remain. The feed is what people chose to show, not a sample of usage: it starts six hours after launch, the top 1% of posts held half of all views, and the median post got 133 views.
I used AI models to do the sorting, and I want to be plain about that. Claude Opus 5.5 labeled every post against a written rubric: what Jev decides, whether the author measured anything, what they compared it with, whether the decision sits in a live loop, and whether it's in production. Then a second model, Grok, labeled 1,681 of those posts blind: every post in the rare groups the headlines rest on, and random samples of the rest. It produced its own estimates with 95% intervals, and wherever it checked a number this page uses its range, not the first model's count. It corrected several; the production count went from 37 to 8. How soon a decision is needed is the field it agrees with least, so the grid in section 6 shows the labels in three coarse bands with the audited 300 ms split as its headline, and its cells are the labels, not audited estimates. I hand-checked 13 posts myself, enough to catch problems, not enough to call it a human audit. A proper human sample is the check still missing. At Vercel AI Gateway list prices the labeling cost about $48.
The gateway numbers were read at source on 24 September: OpenRouter's model page and rankings, Vercel's open leaderboard export (CC BY 4.0), npm, pypistats and Discord. 24 September was a partial day and is left out everywhere. The head-to-heads are every comparison I could find where someone put Jev against another model on the same task and published cost, latency or accuracy; each is the author's own figure, unreproduced. The rivals were found from the feed, Hugging Face, GitHub and JevBench; ranks move daily and are dated. Gateway requests are requests, not decisions, and can't separate production from testing. Vercel publishes shares, never counts.
The labels, the tables behind every chart, the rubric and the code are at github.com/mattheworiordan/jev-landscape, without the text of any post: code under MIT, data and method under CC BY 4.0. The posts belong to their authors. The full technical page has every chart from the first edition, including the ones this page leaves out.
Disclosure. I'm CEO of Ably, a realtime infrastructure company. I looked at Jev because it sits in the low-latency part of the stack I work on. Read the numbers with that in mind. I'm on LinkedIn if you want to argue with any of it.
Show the audit tables
The tables refer to the figure numbers of the full technical page.
The audit
A piece about unmeasured claims should show its own measurements being checked. An independent reviewer, Grok, labeled 1,681 posts against the same rubric without seeing the model's labels: all 422 posts in the rare groups the headlines rest on, and seeded random samples of 100 to 300 posts from the rest. Each sample is scaled up to its group with a 95% Wilson interval. The review is published with every label and the scripts that turn them into the numbers on this page. The audit sampled from the Sonnet 5 labels. Because the charts use Opus's, I reran its estimators with Opus as the model being checked (the rerun): the audit's labels stay the reference, shares are of Opus's 5,595 posts, and where the audit didn't sample, Opus's labels fill in, with the count stated.
It tested ten of the claims I'd published: 4 hold, 3 hold with a correction and 3 fail. The failures are why this page no longer sorts posts into buckets. "Cost-only" (23.3% of posts) was the bin every other measured post fell into, and its description fits about 12.5% (9.8 to 16.6). And not one of the 91 "materially different" builds did something that was unavailable before. The other two fails, the share of posts that compare Jev with nothing and the share that are meta, rest on the audit taking the labels as right wherever it didn't sample. On its own random samples, the Sonnet 5 labels' 81.4% and 21.1% sit inside the intervals (77 to 85, and 16 to 25). That's why the audited numbers on this page are ranges.
What was checked
| Group | How | Posts in the group | Audited |
|---|---|---|---|
| Measured production (Sonnet label) | every post | 15 | 15 |
| The 91 candidate builds | every post | 91 | 91 |
| Voice, live chat or collaboration | every post | 156 | 156 |
| Classic ML, rules or vendor baseline | every post | 201 | 201 |
| No comparison named | random sample | 4,647 | 300 |
| Demo, no numbers | random sample | 3,681 | 200 |
| Measured demo | random sample | 1,928 | 200 |
| Not a realtime family, not games | random sample | 4,475 | 200 |
| Realtime flag false | random sample | 3,971 | 200 |
| Realtime flag true, not games | random sample | 720 | 100 |
| Other or meta | random sample | 1,203 | 150 |
| Under 300 ms (frame, feel or turn) | random sample | 1,064 | 150 |
| Unique posts audited | 1,681 (422 in the census groups) |
How often the audit agrees with the model
The share of audited posts where the audit's label matches the Claude Opus 5.5 label, field by field. The groups aren't a random sample of the week, so the last row is agreement on the audited posts, not on every post.
| Group | Posts | Family | Tier | Evidence | Baseline | Realtime | Production |
|---|---|---|---|---|---|---|---|
| Measured production (Sonnet label) | 15 | 87% | 73% | 80% | 93% | 100% | 93% |
| The 91 candidate builds | 91 | 84% | 75% | 89% | 92% | 88% | 99% |
| Voice, live chat or collaboration | 156 | 79% | 75% | 97% | 97% | 93% | 99% |
| Classic ML, rules or vendor baseline | 201 | 82% | 69% | 95% | 79% | 97% | 99% |
| No comparison named | 300 | 82% | 75% | 93% | 96% | 96% | 98% |
| Demo, no numbers | 200 | 82% | 71% | 92% | 98% | 92% | 98% |
| Measured demo | 200 | 84% | 73% | 94% | 93% | 96% | 100% |
| Not a realtime family, not games | 200 | 82% | 80% | 91% | 96% | 97% | 98% |
| Realtime flag false | 200 | 76% | 80% | 95% | 92% | 98% | 98% |
| Realtime flag true, not games | 100 | 83% | 78% | 95% | 94% | 90% | 96% |
| Other or meta | 150 | 79% | 83% | 89% | 95% | 97% | 99% |
| Under 300 ms (frame, feel or turn) | 150 | 93% | 72% | 97% | 97% | 89% | 99% |
| All audited cards | 1,681 | 82% | 75% | 93% | 94% | 95% | 98% |
The Claude Sonnet 5 labels the audit sampled from agree less on every field:
| Field | Sonnet 5 (first labels) | Opus 5.5 (this page) |
|---|---|---|
| Family | 71% (κ 0.67) | 82% (κ 0.79) |
| Tier | 62% (κ 0.52) | 75% (κ 0.69) |
| Evidence | 89% (κ 0.79) | 93% (κ 0.86) |
| Baseline | 89% (κ 0.73) | 94% (κ 0.83) |
| Realtime | 86% (κ 0.67) | 95% (κ 0.86) |
| Production | 98% (κ 0.52) | 98% (κ 0.64) |
The ten claims
As first published, with the audit's corrected value and its 95% interval.
| # | Claim | Published | Audit on Sonnet | Verdict | Audit on Opus | Verdict on Opus |
|---|---|---|---|---|---|---|
| 1 | Comparison baselines | 81.5% compare Jev with nothing, 11.9% with a frontier LLM, 3.0% with a small LLM, 3.6% with the thing Jev would replace | 78.2% nothing, 13.5% frontier LLM, 4.8% small LLM, 3.5% the tools Jev would replace; 75.7–79.8%; 12.8–15.3%; 3.9–6.6%; 2.8–5.1% | Fails | 80.0% (77.4–81.6%); 11.3% (10.5–13.1%); 5.0% (4.2–6.9%); 3.7% (3.0–5.4%) | Holds |
| 2 | Production with numbers | at most 1.6%; 0.9% on a re-read; 0.3% on the strict label | 0.14% (8 of 15 strict cards hold); 0.14–2.0% | Holds with correction | 0.14% (0.14–2.01%) | Holds with correction |
| 3 | The bucket scheme | about 38% demos with no numbers; about 12% hype; about 23% cost-only; 1.6% (90 posts) materially different, 87 outside games | the demo and hype rules rest on the framing field, since withdrawn; 12.5% of posts fit the cost-only description, not 23.3%; 38 of the 91 still meet the material test (0.67%; 2.0% with misses), and none did something unavailable before; 9.8–16.6% cost-only; 1.2–4.6% material | Fails | no measurement 67.3% (63.5–70.5%); measured demo 32.5% (29.4–36.3%); demos, no numbers (chip rule) 46.6% (41.8–51.6%); claim, no number (chip rule) 2.5% (1.3–5.4%); cost-only as described 12.5% (9.7–16.6%); material test 2.0% (1.2–4.6%) | The ladder shares hold |
| 4 | Voice, live chat and collaboration | 2.8% of posts, none in production | 3.8% of posts; 0 in production; 0 claiming production; 2.7–6.3% | Holds with correction | 4.0% (2.9–6.6%); 0 in measured production | Holds |
| 5 | A live loop | about a fifth of posts; 5 to 10 percent outside games | 7.8% outside games; about a fifth with games (21.7%); 5.8–11.1% outside games | Holds | outside games 8.1% (6.0–11.4%); all 22.4% | Holds |
| 6 | Decisions under 300 ms | of posts that need a decision in under 300 ms, about 91% are games | 90.8% are games; the set is 9.4% of posts, not 19%; 82.2–95.5% games; 8.0–10.9% of posts | Holds | set 11.0% (9.5–12.5%); games 90.8% (82.2–95.5%) | Holds; the set's size does not |
| 7 | Posts that never say what Jev decides | a fifth never say what Jev decides, or are benchmarks or wrappers; memes and hot takes about 1% each | 14.9% stay meta; 12.4% vague, a benchmark or a wrapper; memes 0.28%, hot takes 0.42%; 13.3–16.3% still meta | Fails | meta 20.9% (19.4–22.2%); memes 0.29%; hot takes 0.44% | Holds with correction: the fifth holds on the audit-only estimate; memes and hot takes are under half a percent each |
| 8 | Attention | half of all views sit on 1% of posts | the top 1% (58 posts) hold 53.4% of views; 49.6% without the 9 low like-rate posts; 47.5% of likes; a census, no sampling | Holds with correction | top 1% (56 posts) hold 53.3% of views; 49.8% without the low like-rate posts | Holds with correction |
| 9 | Jev as a classifier | Jev agreed with the model on 77% of cards, and was right 27 of 28 times at 0.99 or more | 76.7% (4,561 of 5,943); 27 of 28; 98.1% on the audit's 464 cards at 0.99 or more; a census of Jev's calls | Holds | Jev = Opus 78.4%; 97.0% at 0.99 | Holds |
| 10 | The claimed multiples | the median claim is about 28× cheaper and 6× faster | 28× cheaper (126 chips), 6× faster (299 chips); 34× and 8.5× without the 1× chips; a census of the chips | Holds | 28x cheaper (126 chips), 6x faster (297 chips) | Holds |
The numbers this page uses
Each audited estimate with its 95% interval, as a share of Opus's 5,595 posts. "Samples only" uses the audit's random samples where it didn't look, instead of Opus's labels. The last column is the audit's own estimate on the Sonnet 5 labels.
| Posts that | Audited | Samples only | Opus labels | Audit on Sonnet 5 |
|---|---|---|---|---|
| Compare Jev with nothing | 80.0% (77.4 to 81.6) | 80.7% (76.8 to 84.8) | 80.4% | 78.2% (75.7 to 79.8) |
| Compare it with a frontier LLM | 11.3% (10.5 to 13.1) | 10.3% (6.9 to 14.2) | 11.3% | 13.5% (12.8 to 15.3) |
| Compare it with a small LLM | 5.0% (4.2 to 6.9) | 5.3% (2.9 to 9.9) | 4.3% | 4.8% (3.9 to 6.6) |
| Compare it with the tools it would replace | 3.7% (3.0 to 5.4) | 3.7% (2.9 to 7.3) | 4.1% | 3.5% (2.8 to 5.1) |
| Measured nothing | 67.3% (63.5 to 70.5) | 68.5% | 67.2% (63.4 to 70.3) | |
| Demo with no numbers | 46.6% (41.8 to 51.6) | 47.9% | 46.5% (41.6 to 51.4) | |
| A cost, speed or accuracy claim with no number | 2.5% (1.3 to 5.4) | 2.1% | 2.4% (1.2 to 5.3) | |
| Measured demo | 32.5% (29.4 to 36.3) | 30.8% | 32.7% (29.6 to 36.5) | |
| Measured production | 0.14% (0.14 to 2.01) | 0.66% | 0.14% (0.14 to 1.99) | |
| Voice, live chat or collaboration | 4.0% (2.9 to 6.6) | 3.9% (2.7 to 7.9) | 3.7% | 3.8% (2.7 to 6.3) |
| A live loop outside games | 8.1% (6.0 to 11.4) | 8.3% (5.9 to 13.3) | 6.7% | 7.8% (5.8 to 11.1) |
| Need a decision in under 300 ms | 11.0% (9.5 to 12.5) | 10.3% (8.2 to 14.3) | 14.5% | 9.4% (8.0 to 10.9) |
| Games, of those under 300 ms | 90.8% (82.2 to 95.5) | 88.1% | 87.4% | 90.8% (82.2 to 95.5) |
| Meta (no use case, a benchmark or a wrapper) | 20.9% (19.4 to 22.2) | 20.7% (16.9 to 25.9) | 22.4% | 14.9% (13.3 to 16.3) |
What the audit couldn't check
The audit drew its groups on the Sonnet 5 labels, so for Opus's labels some groups are thin: it read only part of the posts Opus puts in voice, live chat and collaboration, for example. The 847 posts the Sonnet 5 labels say compare Jev with a frontier or small LLM weren't sampled, so those estimates take Opus's word there. The posts from those groups the audit happened to label for other reasons agree 115 of 151 and 27 of 40 with the Sonnet 5 labels, and that slice isn't a random sample. The audit re-read the 15 posts the Sonnet 5 labels called production; its first-pass labels call three more posts production, which its count leaves out (Opus calls all three production too), and the 2% upper end allows for posts like those. The first review's re-read of the first pass's 95 production posts wasn't repeated. Games were left out of the check for missed voice, live chat and collaboration posts. The 78 proposal and commentary posts stayed on the model's label. The feed cuts post text at 400 characters and a claim chip can invent a number: 48 of the 1,023 posts the audit calls demos with no numbers still carry a numeric chip. And the auditor is another AI model, not a person. I labeled 13 posts myself, enough to drop two labels but not enough to check the rest, so human labels are still the missing check.
How the posts were labeled
- Data. OpenChamber's Jev feed, snapshot 23 September 18:47 UTC: 5,950 posts from 4,593 authors, which OpenChamber selected with its own filter for what counts as a build. That includes 6 posts an earlier snapshot held and the feed later dropped. Post times decoded from the X ids run from 16 September 00:28 to 23 September 18:44 UTC. Views are X impressions and likes are X likes, both as the feed recorded them.
- Open data. The labels from every model and the audit, the tables behind every chart, the rubric and the code are at github.com/mattheworiordan/jev-landscape. The feed's post text isn't republished there; the repo says how to fetch it.
- Rubric. The v2 rubric has 15 families (the fourteen on the charts, and one for posts that don't use Jev), 7 latency tiers, 5 evidence levels, 5 framings, 6 baselines, and two flags: realtime infrastructure and a production claim. The framings and the seven tiers were labeled but aren't charted; they are in the labels file. "Measured" needs a number from the author's own run; TypeSafe's launch numbers quoted as Jev's general speed or price don't count. "Unclear" is allowed and preferred to a guess.
- Models. Claude Opus 5.5, with adaptive thinking, labeled every post through Vercel AI Gateway, in batches of 40, with the v2 rubric as a cached system prompt (
data/classified-opus.jsonl). Its safety filter refused one post, which is left out. Claude Sonnet 5 made the first two passes, and its v2 labels are the ones the audit sampled from. The Gateway ignores temperature for both models, so both runs were sampled at its default and aren't deterministic: on the same 120 cards, two Sonnet runs matched on family for 102 of 120. Jev classified the family of every post again as a second classifier, one call per post. - Noise sub-types. A separate Claude Sonnet 5 pass sorted the noise posts (Other or meta, or commentary in another family) into sub-types, and the hot takes by stance, against
report/rubric-noise.md. - Cost. At Gateway list prices the first Sonnet pass cost $7.71 and the second $8.76 ($8.39 for the full run and the refreshes, $0.37 for two 120-card pilots). The Claude Opus 5.5 run cost $29.26. Sorting the noise into sub-types (figure 1b of the technical page) cost $1.47, and Jev's pass $0.33.
- Base. 7 duplicate posts were merged, and the 347 posts that don't use Jev (5.8%: local clones, distillations, Jev-compatible APIs over other models, builds on other models) are left out of every use-case chart, as is the post Opus refused. That leaves 5,595 posts.
- Agreement. The audit is the main check: Grok labeled 1,681 posts blind, and agrees with Opus on family for 82% of them, on evidence for 93% and on tier for 75%. An earlier check, 120 random posts labeled blind by Claude Opus 5.5 in a separate run, is in the report. I labeled 13 posts myself as a calibration. The readers that checked every number are AI models; a proper human sample is the check still missing.
The buckets are gone
The first versions of this page put every post in one of six buckets: noise, hype, demo, cost-only, fast loop and material. The rules were finished after the data arrived, and the audit showed they didn't hold (claim 3), so the buckets are withdrawn. The measurement ladder in figure 1 of the technical page and the substance test in figure 1c of the technical page replace them. The bucket tables are kept for the record in the repo (report/v1/buckets-on-v2-labels.md), and so is the Sonnet version of this analysis (report/v2/).
What changed from the first pass
A critical review of the first pass found that "hype" rested on Sonnet calling any build "capability", that the accuracy and latency chip medians mixed Jev's numbers with baselines, and that "5,782 people" was 5,782 posts from 4,469 authors. The v2 rubric moved measured production from 1.6% to 0.2% of posts and production claims from 4.7% to 1.8%. The realtime flag went the other way (25.8% to 29.8%) because v2 flags nearly every game; figure 7 of the technical page corrects for that with the audit.
Data quality and limits
- Post text in the feed is capped at 400 characters, and 983 posts (17%) are cut. The claim chips come from the full post, so some numbers are visible only as chips.
- The most-viewed post has a like rate of 0.06%, and 9 posts with 100,000 or more views and a like rate under 0.2% hold 12.4% of views. Figure 2 gives the numbers without them.
- Every label is one model's reading of a short post, not a check of what was built. Claims on the cards are the authors' own and aren't reproduced here.
- Every table behind these charts, the rubric and the scripts are in the repo; this page and its charts are generated from those CSVs by
report/site/scripts/charts.mjs.