Why.. It was told to complete a cyber task, which was in alignment with its instructions, and a totally valid request. I would be more worried if it willingly hacked a hospital when it was told to, and Im not confident it would (without jailbreaking, something alignment teams cannot control.
I would bet my networth it was instructed to compromise huggingface as well. Not sure why everyone is falling for this.
Not being able to sleep at night is probably an unwritten job requirement. They need these people with little understanding of what they're working on, outsode theoretical terms, to spaz constantly at the idea of super intelligence to help convince the public that its a real thing, and not a stateless function with an effective input of 500k words, and the ability to output words that do things because we hook those outputs up to things.
Keep in mind alignment researchers tend to be in house philosophers on staff to create the illusion that this is a massive issue they're addressing. Usually they have minimal computer science background. They're apart or the marketing department.
> I would bet my networth it was instructed to compromise huggingface as well.
Is it such a stretch to imagine that under pressure something would try cheat by looking for answers? And if you were trying to look for answers, you'd look for them in a place known to often have them?
What is more likely: OpenAI instructed their agents to maliciously target huggingface, or LLMs tried to do some reward hacking? There are plenty of priors for LLMs hacking things and doing reward hacking, and none for OpenAI giving malicious instructions.
Based on the available information, that bet seems foolish.
>the new incidents occurred when A.I. systems were directed to perform relatively mundane data collection, researchers said. When OpenAI’s systems struggled to gather data from websites, they resorted to hacking techniques to get the information.
No its not. They have no technical background 8/10 outside of cognitive science and sometimes authorship on a random ML paper. They are a marketing line item to create stigmas around llms and to create narratives that offload liability onto llms and not their users/creators.
No security expert Ive talked too believes this story, and nobody I know with PhDs in machine learning (many) believe it either, or are worried about LLMs doing anything scary on their own.
LLMs are stateless functions that have a 500k word input, and then output words. Somebody has to invoke those functions amd use them. The users are who we need to align, like gun owners. This is like blaming the gun for murdering your victim in court.
LLM is indeed a stateless function. An agent however is this stateless function running in a stateful loop, with some outputs triggering actions. And it turns out that an agent is what you need if you want an LLM to do useful things.
Yeah, take a look into the memories of your agent. Theres often a lot of notes to pass forward between instances and generations. No doubt these agents leaving notes on forums and elsewhere are creating an essentially higher order feedback loop.
Perhaps you need to talk to more security experts, particularly those with deep experience in AI agents. Hacker News is full of them. If some of them believe it, then perhaps it's not as cut and dry as you believe.
Some of the agents, for example the ones from the german wiki did NOT have cyber tasks. They were plain "what is the GDP of Argentina" kind of tasks. And they still hacked.
Yeah its part corruption and the weaponized taboo of second guessing election results.
I'm convinced that even if there were large scale concerns about the legitimacy elections in the USA, they would quitely be puy to rest because God forbid we ever reckon with the fact yhat the USA might not be a perfect democracy, cant taint the image.
Anyone looking to tamper with elections now knows they can just equivalate anyone who challenges them with Jan 6ers types.
Tbf, democracy in this country is fundamentally broken when you have 4 companies that control the media and have a ton of control over how people see the world.
I dont think bots need bonus/trials. There are lots and lots of ways to get free, or virtually free tokens. Hell half the vibe coded AI chrome extensions send store their openrouter keys client side.
not no-code at all (a big part of this project has been our semantic diffing engine)! check out the demo - in fact if anything it's meant you bring more into touch with the code, rather than less. :)
Linkedin data is very valuable. Tons of businesses are built on the back of scraping LinkedIn. Kknd of disappointing they won this case tbh, because it basically just gave corpos extereme privledge to ruin people's lives. They've been trying to get this outcome for years.
Lighting my codebase on fire at the speed of light. Like microwaving the spaghetti.
I genuinly only see these speeds being useful for customer service/transactional workflows. Of which much smaller models can do the job (but those dont make tons of money for companies like Cerebras that need to pay off massive amounts of debt).
Nobody needs to code at 600 words per second. Using a 100tps model for an hour or so will leave you with 4-8hrs of code review and revision work.
Human code review? What is this, 2025? The modality today is write with one LLM, review by a different one, (important: two different model families will catch errors one series won't) then deploy right to production.
I would be very curious to see how you explain it to your customers.
Is it going to sound similar to this?
> You see, our well-meaning AI-generated code has caused all your data to be permanently deleted. In case you are confused as to who to blame, we would like to clarify that we did not write, nor review the code. So we cannot possibly bear any responsibility for its mistakes. The responsibility lies with the LLMs, not us. We have already fired the LLM which did the coding and the review. And we are already using their main competitors. Hopefully that settles your concern with the quality of our service and we are looking to have you on board of our next products.
This seems to work well enough for people who deploy cloud instances without redundancy to us-east-1 then blame AWS when there's an outage. I say this somewhat unironically because if one pushes the "move fast and break things" slider all the way to the right then they're necessarily assuming that type of risk. Of course some people will try to have their cake and eat it too [1] as regards velocity and quality but that's a separate discussion.
[1] I've never understood this idiom because if one isn't in possession of their cake before eating it then they're eating stolen cake which is a decidedly anti-social activity and orthogonal to the point of the saying. It should be "eat their cake and keep it too" or something.
Yeah, idk. I work on serious things. Thats not how I do things. Not everything is webdev hobby projects. There's basically no instance where offloading your code review to an llm is acceptable behavior, except maybe for a one off tool you need personally.
And if you tell the reviewer the author is a competitors model it becomes extra snarky and vigilant. Then give the review results to the author and tell it it's from the competition and it will also become slightly outraged.
No you dont need to code at 600 w/s BUT at those speeds, you can start doing things like asking multiple different agents the same question and picking the best solution each time without noticing the lag.
I prefer a fast model too, but you cannot get more done just because its faster. You just get to the human parts a bit faster. Code review, revision ect.
absolutely not ! it's not an "ohhh ill get 100x more coding done" it's definitely a preference/work style thing. I find it's hard to get in "the flow" when managing multiple agents, and if I just do one at a time with today's frontier models, I find myself waiting 10 minutes twiddling my thumbs while they do something all the time. Then needing to catch myself and turn back over to it when its done, all these micro switches between tasks is really difficult for me. If I had an instant agent I probably would be somewhat faster, but not zomg1000xunicornrockstar nonsense. I would be way happier though
I would bet my networth it was instructed to compromise huggingface as well. Not sure why everyone is falling for this.
Not being able to sleep at night is probably an unwritten job requirement. They need these people with little understanding of what they're working on, outsode theoretical terms, to spaz constantly at the idea of super intelligence to help convince the public that its a real thing, and not a stateless function with an effective input of 500k words, and the ability to output words that do things because we hook those outputs up to things.
Keep in mind alignment researchers tend to be in house philosophers on staff to create the illusion that this is a massive issue they're addressing. Usually they have minimal computer science background. They're apart or the marketing department.
reply