A given model may or may not have strong self-awareness with respect to how strong it is with different tools.
If it's unusually skilled with one tool, but it's not the default tool for a job, then you have to put it in their hands before they reach for something else.
The Opus 5.5 javascript-art + art direction definitely seems to be one of these surprise capabilities jumps. Maybe strong enough that providers will start to nudge in that direction in the system prompt, so that the user request doesn't need to specify it.
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
I am saying that if they lost alignment and their new model started injecting cryptolockers, dropping all tables of productions databases into the software or if it stated writing poisonous cooking recipes there would be backlash, lawsuits, and more towards such a lab. Consumers don't want to trust such a dangerous model so they won't buy the tokens and the lab will not want to spend money on lawsuits.
>Or you don't think that sufficiently advanced AI can cause (mass) harms?
Even a feather can cause mass harm if it's used to sign a declaration of war. Something merely being capable of causing mass harm is not an issue and doesn't mean that the existence of feathers are an issue.
But this assumes an evil superintelligence. We don't even need that for terrible things to happen; we just need people doing people things and AI doing AI things at scale, and that scale is rapidly exploding as we scramble to deploy AI in the real world.
Case in point, that hallucination (which, if it was an evil superintelligence, could have been "strategic") that almost led to US boarding a Chinese ship over suspicions of nuclear weapons: https://www.msn.com/en-gb/news/other/us-military-ai-failure-...
It doesn't have SkyNet, it just has to be WOPR. And it doesn't have to be people who are motivated to cause harm, it just has to be people making consequential decisions who are careless, distracted, paranoid or anxious -- which everybody is at some point or another.
And I almost kill people when stopping at a cross walk. That doesn't mean I harmed a pedestrian. Society is set up to be very robust. Humans themselves make mistakes and do bad things and society has had to learn how to live with that truth.
>it just has to be WOPR
Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
> And I almost kill people when stopping at a cross walk. That doesn't mean I harmed a pedestrian. Society is set up to be very robust. Humans themselves make mistakes and do bad things and society has had to learn how to live with that truth.
Maybe not you personally, but many, many other people have killed many, many pedestrians, and when they exhibit a pattern of bad driving -- or other deviant behavior -- we take them off the streets. That is an example of society being robust.
Except, over here, we're rushing to make AI, which we know has many deviant behaviors and we know caused harms in many circumstances (starting with AI-assisted suicides), even more powerful AND deploy it in more and more real world systems!
As history and, literally, current events show us again and again, society has failure modes that lead to widespread harm and destruction. We've had world wars and then literally had multiple close calls with nuclear war right after. And then we have all that's going on out there. (In related news, the Pentagon threw a hissy fit because Anthropic would not let them use Claude for autonomous killing machines. Do I have to even mention what they've been up to these days?)
Most of these failures are caused by misaligned incentives and socioeconomic forces. The incentives and forces around AI have hints of many brand new failure modes that we can't even foresee because things are moving so fast.
> Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
"We're already going down this mountain pass at 200 miles an hour, why slow down now?"
I think we should not just slow down frontier AI development, we should also slow down where and how that AI gets deployed.
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
Batteries and solar and wind are a lot cheaper, faster to deploy, and scale a ton better.
Nuclear is far too slow to solve our emissions problems, there's no chance of us being able to scale the nuclear industry to be able to confront climate change on the time scale necessary.
In contrast, batteries, solar, and wind are scaling on their own, without the massive government subsidies that would be needed to deliver nuclear too late to help.
The app has recently moved into production, so I'd encourage anyone with verbal but pre-literate kids to check it out.
The basic pitch is high efficiency acquisition of the highest yield phonetic mapping skills, and nothing else. I myself am something of a screen-time zealot and very wary of applying engagement mind hacks against kids. The narrow focus allows for good progress on a very modest schedule (recommended cap at n minutes per day for n years old, n >= 2). I defer the social and cultural aspects of learning to read entirely to parents.
It is mostly intended for parent-child co-use, although kids with a bit of experience can drive many of their own sessions most of the time.
When it stops with the compacted isomorphic plagiarism using other peoples work like an intelligence campaign. LLM are not "AI" in my opinion, but are good at domain context search. Neuromorphic computing may change that one day, but it will unlikely arise from the LLM cults. =3
As I understand it, the rough guess as to what's happening here is that most recent capabilities progress comes from specific verifiable-rewards reinforcement training (RL). The RL pressures are all about task performance, but (surprise surprise) highly human-legible English language usage isn't very important to the models abilities to address the tasks.
Weirdly enough, the pressures are having them drift toward novel dialects of English that work well for their own chains of thought. Open question about whether they'd drift all the way to a new language given enough time.
This is tricky, because we really want language-independent training of skills. We know that self-play type of reinforcement learning is incredibly effective when possible. But at the same time, they are our tools - so we need supervised language training for this reason? It's possible that training just needs to be rebalanced so that RL with rewards is balanced with rounds of language adjustment. And to really make that happen, benchmarks need to score the models on that.
Dario signed the Pacing the Frontier open letter when Fable/Mythos seemed from the outside to be an insurmountable lead.
Also he's been saying versions of this day in and day out for as long as he has had anyone's ear.
It's possible to read that his "strategic" value of this statement is higher now than it was 10 days ago. But that doesn't change anything about his consistent, long standing, positions.
If it's unusually skilled with one tool, but it's not the default tool for a job, then you have to put it in their hands before they reach for something else.
The Opus 5.5 javascript-art + art direction definitely seems to be one of these surprise capabilities jumps. Maybe strong enough that providers will start to nudge in that direction in the system prompt, so that the user request doesn't need to specify it.
reply