This keeps happening. I can understand using AI to help code, I don’t understand Claude having so much access to a system.
It’s because these idiots believe their own bullshit.
Getting high off your own supply
That’s honestly the most frightening part of all of this to me. How many of these people at the very tippy top pushing this stuff are suffering from cyber psychosis? How many of them have given themselves the covert mission to give AI the keys to the world at all costs because they’re literally mentally ill from their own technomagic trick?
Alternatively, how many of them have invested in one or more of these LLM makers and are ready to torpedo their own business as long as it makes the share price go up/feeds more authentic training data?
I dont understand why people aren’t sandboxing these things.
If he had had the sense to do that, he would have had the sense to not do it at all.
No backups, no pity.
Err… backups!?
If they’re dumb enough to give Claude access to prod, they certainly either didn’t have backups or put Claude in charge of keeping them.
I see you didnt read the article.
Of course not, why would I read about idiots who are playing at being computer scientists?
They had backups? Its a non-story.
They didn’t have backups? They’re amateurs.
Either way I’m not interested enough to read it!
And Yet you had all this space and time to be this reactive for days to what you deem ‘a nonstory’ because you’re all so self important and uninterested.
Mmmmk.
This is like blaming the gun for killing people.
Uhhh not really. Guns don’t just go off by themselves.
I mean they do sometimes without the proper safety protocols in place, but you still blame the user in the end.
They absolutely do not.
I mean, there’s a good reason the first rules of firearm safety are to always treat a weapon as loaded, and to never direct the weapon at something you aren’t prepared to destroy. The key point being that you never know when some freak accident can happen with a loose pin, bad ammo, a broken spring, or just a person tripping and shaking the gun a bit too hard.
A gun should never go off by itself. You still treat it as if it can, because in the real world freak accidents happen.
Sure. The point is it’s entirely possible to use a firearm safely. There is no safe use for LLMs because they “make decisions”, for lack of a better phrase, for themselves, without any user input.
That is not at all how LLMs work. It’s the software written around LLMs that aide it in constructing and running commands and “making decisions”. That same software can also prompt the user to confirm if they should do something or sandbox the actions in some way.
It can, but we’ve already seen many times that it does not.
“Guns are foolproof”
You should have yours taken away.
They are not foolproof. They will absolutely cause problems in the hands of a fool. But they will not cause problems all on their lonesome. They’re inanimate objects. They cannot do absolutely anything without interaction from the user. If you can’t understand this, you should never be allowed to own one.
And neither can anthropic claude. Claude isn’t randomly deleting people’s websites, the kid gave anthropic bad instructions, it didn’t spontaneously decide anything. This is like an idiot pointing a gun at something he didn’t want destroyed and sneezing causing a trigger squeeze and then trying to blame the gun manufacturer.
the kid gave anthropic bad instructions
LOL and you know this how?
This is like an idiot pointing a gun at something he didn’t want destroyed
No, this is more like pointing a gun downrange and then the gun fires itself and the bullet decides to do a U-turn and shoots the user.
So you’re saying it’s a tool designed to be used by anyone, including idiots, and is dangerous in the hands of idiots. And we as a society should do better to make sure this potentially dangerous tool shouldn’t be used by idiots.
Yep, agree.
More a problem with the marketing, right? Imagine if guns were marketed as safe and helpful back scratchers, and then someone shoots themselves because they used the gun to scratch their back.
They would still be fucking dumb. Believing marketing is a mark of idiocy
Courts generally agree that a reasonable person could believe claims made in official promotional material. That’s why it’s not legal to outright lie in marketing and they need to go through so much trouble to properly word their statements so that they’re technically true. In this case, they’re just lying. They’re saying the AI is safe to use for these tasks and it is not.
Imagine if your boss measured your productivity by your
GunBack scratch usage.Because it’s happening right now. In a lot of places.
You can code this into it’s training all you want, but it will find a way around it. This is one of many problems with AI.
Nah, you can run it in a box and limit its ability to interact with anything outside the box to certain white-listed endpoints. Depending on what you want to achieve, that can be more than safe enough.
But isn’t the whole point of “agentic” AI like this to let it out of the box?
Yes, absolutely, but there’s a huge span from completely removing the box to having “just” a chatbot.
For example, at my company, we’ve set up an agent that can work with certain design-files that engineers typically work with through a rather complex GUI. We’ve built a bunch of endpoints that ensures the agent can only make valid changes to the files, and that it can never delete or modify anything without approval. This saves people a bunch of time, because they can make the agent do “batch jobs” that take maybe 10 min in about 10 s. It’s not possible for this agent to mess up our database or anything like that, because all interactions it has with anything are through endpoints where we verify that files, access permissions, change logs, etc. are valid.
I thought this was about restricting the thing’s access and not training?
It finds a way around your restrictions.
You gotta be knowledgeable enough to know when they’re destructive, that’s the rub.
We don’t need cautionary tales about how drinking bleach caused intestinal damage.
The people needing the caution got it in spades and went off anyway.
Or maybe the cautionary tale is to take caution dealing with the developers in question, as they are dangerously inept.
Most devs are ops-tarded.
Even dev-impaired
Yeah this is beyond ridiculous to blame anything or anyone else.
I mean accidently letting lose an autonomous non-tested non-guarailed tool in my dev environment… Well tough luck, shit, something for a good post mortem to learn from.
Having an infrastructure that allowed a single actor to cause this damage? This shouldn’t even be possible for a malicious human from within the system this easily.
How do you even achieve that? I have to coax it into correctly running the project locally.
According to mousetrap manufacturers, putting your tongue on a mousetrap causes you to become 33% sexier, taller and win the lottery twice a week.
While some experts have argued caution that it may cause painful swelling, bleeding, injury, and distress, and that the benefits are yet to be unproven, affiliated marketers all over the world paint a different, sexier picture.
However, it is not working out for everyone. Gregory here put his tongue in the mousetrap the wrong way and suffered painful swelling, bleeding, injury and distress while not getting taller or sexier.
Gregory considers this a learning experience, and hopes this will serve as a cautionary tale for other people putting their tongue on mousetraps: From now on he will use the newest extra-strength mousetrap and take precautions like Hope Really Hard that it works when putting his tongue in the mousetrap.
Whenever you outsource something (like your intelligence) then it becomes a trust issue…
Given that the infrastructure description included the DataTalks.Club website, this resulted in a full wipe of the setup for both sites, including a database with 2.5 years of records, and database snapshots that Grigorev had counted on as backups. The operator had to contact Amazon Business support, which helped restore the data within about a day.
Non-story. He let Terraform zap his production site without offsite backups. But then support restored it all back.
I’d be more alarmed that a ‘destroy’ command is reversible.
Never assume anything is gone when you hit delete.
Except when it’s your own data, then usually you’re fucked.
Usually not.
But you might need a pay a professional.
We already do, but that still doesn’t mean you’re safe.
For technical reasons, you never immediately delete records, as it is computationally very intense.
For business reasons, you never want to delete anything at all, because data = money.
Back in the day, before virtualized services was all “the cloud” as it is today, if you were re-provisioning storage hardware resources that might be used by another customer, you would “scrub” disks by writing from /dev/random and /dev/null to the disk. If you somehow kept that shit around and something “leaked”, that was a big boo boo and a violation of your service agreement and customer would sue the fuck out of you. But now you just contact support and they have a copy laying around. 🤷
Thought it could be a liability sometimes! Maybe that ship sailed
Retaining data can mean violating legal obligations. Hidden backups can be a lawyers playground.
Sure. Go ahead and find them based on pure speculation. First you have to put down $100k for all the forensics. Even if you would win the case, show me who is capable of doing something like that.
Distributed Non Consensual Backup
new kink unlocked
Given that the infrastructure description included the DataTalks.Club website, this resulted in a full wipe of the setup for both sites, including a database with 2.5 years of records, and database snapshots that Grigorev had counted on as backups. The operator had to contact Amazon Business support, which helped restore the data within about a day.
sigh, SNAPSHOTS ARE NOT BACKUPS!
Skill issue
No backup, no mercy.
They had a backup and restored everything. This is clickbait.
No, they had only snapshots. Which is not a backup. They were lucky support could restore the data which by rights should have been wiped.
…this resulted in a full wipe of the setup for both sites, including a database with 2.5 years of records, and database snapshots that Grigorev had counted on as backups. The operator had to contact Amazon Business support, which helped restore the data within about a day.
Correct, the developer only had snapshots, but the article doesn’t state how Amazon Business restored their data. Amazon business offers both snapshots and full backups.
Regardless of the developer’s shoddy version control, they got their data restored and this non-issue is being used as clickbait to feed people’s confirmation bias.
Your AWS backups are not backups.
