Pigford ran 10,000 HN comments through Jev, 43% corrected
A new model spent a day on founder timelines, and the interest is about classification, not chat.
Josh Pigford ran over 10,000 Hacker News comments through Jev, a new model, and reported that 43% of them were correcting someone. He then shipped a "well, actually" leaderboard, crediting @dotpem for the suggestion.
43% of them were correcting someone.
That was one of several Jev demos doing the rounds the same day. Yohei tried graph extraction with it, scoring every word for semantic significance from 1 to 5, tagging relevant words with an ID, then building a graph from the high scorers. KP said he was blown away by the demo and noted he is unaffiliated. Guillermo Rauch shared a link. Theo was less impressed, replying to a @kenwheeler tweet about it with a paid promotion jab. levelsio just asked why everyone was talking about Jev today.
What people are actually excited about
Not chat. Aaron Levie's list is the tell, and it is all plumbing. Data classification, routing decisions inside a workflow, decision making on a domain problem, quick judgment calls about safety or security. He called those the gates in a large number of processes and said the approach could be cool in agentic workflows in the enterprise.
Being able to process information insanely quickly, at crazy low costs, with high levels of capability is huge for a wide number of enterprise tasks.
Shreya Shankar put the business logic underneath it in one line. AI functions get popular in databases because nobody wants to do the MLOps work of training their own classifier. That is the same reason a two person software company pays frontier prices for a call that decides whether a support ticket is a refund request. Training a small model is cheap in theory and a maintenance job forever in practice.
The business read
Every data point here is a demo. Pigford's 43% is a fun number from a weekend project, not a benchmark, and nobody in these posts is reporting Jev running in production, at volume, with a cost line attached. The claim being made is speed and price at decent capability, and none of the founders talking about it have published what they paid.
So the useful move is not to swap anything out today. It is to go look at where your product makes per call judgment calls, the classify, route and score steps buried between the parts users actually see, and find out what those cost you right now. If a cheap fast model does handle that middle layer well enough, those line items change shape, and you want your own numbers before the timeline tells you how to feel about it.
Being able to process information insanely quickly, at crazy low costs, with high levels of capability is huge for a wide number of enterprise tasks.
AI functions are popular in databases because no one wants to do the MLOps work of training their own classifier