It's hard to say without knowing their architecture, but I'd guess something like block attention. You can process the prompt separately from the classifications into a latent space and then do some kind of late interaction with the encodings from the classifications.
There are plenty of other ways to do zero shot classification that would result in more "token usage" (really just having to reprocess everything for each class), but the pricing and the way they describe it narrows it down somewhat.
I'm having a really hard time wrapping my head around why Jev is getting so much hype. It feels manufactured to me. I don't think they've proven a significant market for their product, and there's no independent benchmarks that prove anything. To me that's doesn't pass the smell test.
But if they had to show how well their product worked they might give away the whole game... because they'd have to compare their "noul" class against an NLI benchmark for instance, and possibly show they're losing to cross encoders and give away the fact that they are just rebranding NLI. Or rerankers (choice) or zero-shot classifiers.
The AI hype cycle is always looking for the next big thing. It really doesn't take much for enthusiasts to get very excited and push something into the stratosphere. Just not having a vibe coded website, and someone that made ChatGPT is enough to set them apart. Hitting pain points like pricing and speed and also implicitly mentioning llms (even if it's to mention it can't generate text in contrast to llms) make it seem like a major step-up.
I'm not sure anybody but people inside the company know if the model itself is a contribution. There's no publication and no architectural details. There's no benchmarks or comparisons published. You can do all of the things they claim with an LLM, not that I think that's what they did.
Likely they have some encoder (eg ModernBERT) trained to do late interaction or latent states along the lines of ColBERT, Perceiver IO or poly-encoders.
No. Frontier AI companies need the government to shield them from becoming commodity. That's why they're begging for regulation.
On the other side, the current administration also has powerful backers whose businesses are complemented by AI and would benefit from it becoming a commodity. Think about who Vance is speaking on behalf of.
So the AI companies need to use fear to drive public demand for their regulation efforts. IMO there's little to no honest discourse and a lot of attempts to manipulate public opinion for objectives that we have little stake in. I personally don't trust any of the involved parties to make decisions for the public good.
The problem is that at the business level and at least for a large part of the US government, we need to assume that there are no 'good' actors.
Anthropic needs regulation in order to prevent AI from becoming a commodity. This of course does not benefit all tech businesses equally, especially those that are not currently at the AI frontier. So when JD Vance talks about AI, he talks using the mouth of Peter Thiel who may not see benefit from the same policy as Altman or Amodei.
The rest is just public support posturing and most of that is bullshit meant to distract from the high rollers game of winners and losers. The philosophy is money and power, who gets it and who doesn't. Us normies aren't really participants in the game, except where we are being manipulated into cheering for one side or another, and with little stake in the outcomes (although selfishly, I'd be pissed if I didn't have open models to tinker with).
Open source is fundamentally a vehicle for commoditization. This is great if your business is not AI and your business is instead something like GPU hardware or some product that uses AI. But it means that eventually, selling AI is not going to be the money maker.
OSI proliferated open source on a business strategy called "commoditizing your complements". These big companies don't do it out of benevolence. It was pitched to them in a way that FSF did not (which was more about morals and ethics, something business care little about), and it caught on. And the software business became about ads, consulting and cloud services instead.
It's also clearly the wrong question for this decade. Private / public sector cooperation and all of the barriers between them that have been degraded and chipped away has left us with a very broken economic system that looks less and less like capitalism every day.
One only needs to read Stevens' dissent in hindsight to recognize that all of his concerns were and are legitimate, and the worst of which have come to pass.
There are plenty of other ways to do zero shot classification that would result in more "token usage" (really just having to reprocess everything for each class), but the pricing and the way they describe it narrows it down somewhat.
reply