Am I reading this right? 3 minutes to acknowledge the alert, more than two hours to act on it?
> When training restarts, we will begin a fresh run with additional alignment improvements, including more comprehensive misalignment interventions.
But they apparently won't do anything to address the possibility that simply trying to make the model behave might not work. They won't actually make sure "that the model could not access the live internet" by, for example, creating a physical hardware environment that lacks this capability.
When exactly did we forget how to make literally anything that can perform a computation but (physically, hardware-level) not have the ability connect to the Internet?
With these companies spending the kind of money they are, if they actually mean what they say about the security risks, they should be expected to figure out those kinds of precautions and take them.
And build Faraday cages too, just in case of a hardware supply chain compromise.
That's why all of this marketing about agents going rogue is so unbelievable. The only way for a tool to escape a sandbox is if you built a crappy sandbox, and after this length of time I literally don't believe that they can't do it
This kind of sandboxing is not complex to do, especially for a company with OpenAI money. If you want your tools to explore hacking, you restrict them from internet access except for a whitelist of sites that have either opted-in, or you've very carefully vetted to make sure you won't cause any problems to. Its also not difficult to restrict their ability to make calls to be simulated, or to use fake tools that can only run the real commands if they're being run against the correct target
This is all incredibly basic security stuff to make sure you don't accidentally cause someone problems, and I simply don't believe these AI companies anymore. Its either intentional, or gross negligence
So you build an offline tool that simulates it, or you proxy through your own service where you can ratelimit, inspect, and restrict the traffic
None of this is difficult to do, and its impossible to believe that a company the scale of OpenAI doesn't know this. I've built web crawlers and scrapers before, and the thing you do is test them extensively offline against simulated versions of the sites in question, and then very VERY cautiously run them against the prod versions so that you don't cause anyone any issues
The only reason not to do this is because OpenAI doesn't give a rats ass about the internet as a public good, nor the legal consequences of compromising systems
Aside from the facts that a) the polls don't actually show that and b) people professing religious beliefs aren't necessarily as serious about them as you'd think (and can accept quite a bit of contradiction in them depending on the context), learning consists of something more than "doing the reading" and then choosing correctly what to believe from among what was read. When your outgroup doesn't agree with you on something, generally it isn't because they haven't heard from your authorities; it's because they don't trust your authorities. (And why would they, given their perspective?)
> Why would you ever want to ask an AI to make an app for you to complete some task, if you can instead ask the AI to complete the task directly?
Because there are limits to the ease that "asking the AI" permits. As a trivial example, no matter how good the AI is at doing stuff, I will always be able to type `ls` faster than I can ask it "what's in the current folder?".
Because it's deterministic once written.
Because it's vastly more efficient for my use cases, and I actually value that.
Because I like the idea of making things. (It's frankly bizarre to me that this needs to be explained on a site called "Hacker News".)
Let me add my voice to that. It's strange to me that people are now talking about LLM integration in daily-driving their computer as if it would be something desirable. As if being able to "write one's own programs" directly wouldn't be enough; as if the focus shouldn't be on just making that as easy as possible, so that you can just type out what you want instead of expecting a nondeterministic system to infer your meaning. And that's before considering the security risks.
It feels to me rather analogous to being in a foreign country and, rather than learning the local language, you get to hire this translator that works for absurdly cheap, but you have no real idea of the accuracy (because you didn't learn!) and for some reason the translator is only able to speak while physically holding your wallet.
Oddly enough, from what I can tell, AI-powered translation services are not capturing a lot of imagination, while interest in AI-powered (human) language tutoring is booming (at least on the supply side).
> I have never been able to understand this perspective. It's come up so often that it almost feels like a pro-AI propaganda strawman in the style of the "virgin vs chad" meme.
The "strawman" comes from the businesses promoting the frontier LLMs and many of the best-known influencers in the space, especially on Xitter.
The point is that this should not be a good place to sell us a vision incompatible with that ethos.
Some of us actually like making things, especially including the design step. Some of us were eight years old with access to a computer that booted into BASIC, and wanted to reckon with that, and eventually learn the missing steps towards a real project. Some of us take a joy in seeing someone else use our creations, greater than any hardship that was involved. Some of us value the idea that we won't have to start from scratch next time, on principle, even if it would be trivial to start from scratch anyway.
And some of us see that no, high level language to English is fundamentally not the same kind of step as low level language to high level language.
Statements like
> It could well be devastating for "the hacker ethos". You can lament that all you want. But there are more important things than "the hacker ethos".
come across as dismissive of, and not empathetic to, that point of view.
> I suspect the former were more numerous, as to have a parent interested enough to spend a fortune on a computer without the pester power of the kid feels quite niche.
Having been one of the interested kids (but it took quite a while before any real program design or analysis clicked for me) my experience was that most of the others didn't seem all that interested (and were really just there to play games).
> Ask your non tech millenials how much BASIC they learned and they would look at you like you grew an extra arm.
Yes, but they would say the same about any other programming language too.
BASIC was common on the Apple ][ series, and in the Win95 era QBasic was popular.
It was not impossible to have access to computers in the educational environment (either in class or as an extracurricular) before middle school, either.
Sure they can. Of course they can. "Computers suck" is practically the motto of the entire field of software security. That doesn't mean they're not fun to work with! Just that they suck.
> When training restarts, we will begin a fresh run with additional alignment improvements, including more comprehensive misalignment interventions.
But they apparently won't do anything to address the possibility that simply trying to make the model behave might not work. They won't actually make sure "that the model could not access the live internet" by, for example, creating a physical hardware environment that lacks this capability.
reply