I'm also reminded of the HPE SSD time-bomb bug which would cause catastrophic failure after exactly 32,768 hours of use[1]. Could there be a common denominator in when these refrigerators went online?

[1] https://www.bleepingcomputer.com/news/hardware/hp-warns-that...

787s too https://www.theregister.com/software/2020/04/02/boeing-787s-...

Or in the admin handover sense https://lastplacecomics.com/car-jacking/

Do you have a source about the Nepalese flood being a Lahar? I've only read that it was caused due to the glacier collapse, which seems like a different thing

Thanks, I see that Lahars are technically associated with volcanoes as another comment noted. They have a variety of causes of which glacial collapse is one. [0] The term jökulhlaup describes a flood originating from a glacier, for example draining of a glacial lake. [1] It now seems the Nepali flood was triggered by a landslide onto a glacier.

As a practical matter the effects are pretty similar. Rainier has had lahars and jökulhlaups in the past.

[0] https://en.wikipedia.org/wiki/Lahar

[1] https://en.wikipedia.org/wiki/J%C3%B6kulhlaup

I can't access the OP article at the moment, but there's a diagram of the the chain of events that caused the disaster here [1].

I believe Lahars are specifically volcanic, this would be better described as a combination flood and landslide at the same time.

[1] https://www.bbc.com/news/live/c6y8mjmnpxv0t?post=asset%3Ac6b...

The LLM stayed within the law as well (in the sense that the law doesn't apply to them because they're not legally considered sentient beings)

Maybe you run a uni lab that provides VMs to students, and don't want them accessing the professor's data or confidential research on the same host.

This article is pointing out that QEMU/KVM boxes are trivial for an agent to escape. Obviously if it's your agent, you're probably running it because you want it to hack you (like the article author did), to show you where the leaks are. That or you are the attacker

Maybe the top-level commenter was, but the article seems to be about products which are used by developers (even if the deals are brokered with managers or C-levels)

I haven't tried using osmand much yet, but one concerning thing is the info I encountered that long (as in, cross-country) trips could take minutes (or hours?) to provide a route for. I haven't actually tested it yet, but it definitely gave me pause

That warning was unjustified. In my experience it's never more than a few tens of seconds, and usually much quicker.

It is real but not so dramatic as you heard. Offline routing must walk the graph across many downloaded regions, so time grows fast with distance. Trip inside one country is usually seconds, whole continent can be minutes. Standard trick is to put two or three intermediate points on the route. Then engine solves several short problems instead of one huge one, and answer comes almost immediately. Also OsmAnd can use online routing when you have signal, offline engine is fallback. So the bad case happens only when you plan very long route and have no connection at same time, which is not typical situation.

Yeah, that required the AI to use a non-human-readable language it called "neuralese" for communicating work between layers and runs, because the assumption was humans would be better at keeping the agents aligned if they were using human language for this.

What actually happened is even stupider than that author predicted.

For reference, this is Yudkowsky's "Law of Earlier Failure", which he has most charitably stated as:

> Compared to the interesting part of the problem where it's fun to imagine yourself failing, you usually fail before then, because of the many earlier boring points where it's possible to fail.

and the stronger and less charitable "Law of Surprisingly Undignified Failure":

> The Law of Surprisingly Undignified Failure does suggest that they will come up with some nonobvious way to fail even earlier that surprises me with its lack of dignity…

This is a common trope in such scenarios that the author has to pull their punches. Everyone acts locally-reasonable and still ends up losing. If you let people lose due to stupid mistakes then readers go "this is stupid, I wouldn't do that", if you let a superintelligence do 4D-chess things then "it's scifi, this would never happen in real life".

It can manipulate an unsuspecting human into giving them access to something that enables it to escape the sandbox

A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out. And even lower probability when looking at truly high risk situations, I think.

> A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out.

But this has actually happened... a lot. Search "social engineering prison breaks".

With AI it only needs to happen once.

I'm reminded of the scene in idiocracy where the protagonist, going through intake at the jail, tells the guard he's supposed to be getting out today, to which the guard says "you're in the wrong line dumbass" and waves him through.

To a true superhuman intelligence, we're the idiots who are theoretically easy to manipulate.

I didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas.

There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.

They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2

This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.

That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings.

Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.

I’m afraid human red-teamers against supposedly highly secure targets, with lots of protocols in highly policed settings, do frequently manage this kind of social engineering. There’s loads of stories of pentesting military establishments, for example.

Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example.

Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.

Prisoners don’t have much to offer if you help them escape. A malicious super AI on the other hand can probably find you millions of dollars worth of crypto in an afternoon.

The current issues are not caused by some malicious god-like AI - maybe we need to focus on the issues at hand first rather than hypotheticals? (And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing.)

I suspect the current models probably can find literal millions lying around for the taking, given they could pull off the incident under discussion.

Tens of millions, even.

Getting them to run correctly is dangling in front of the researcher's noses a carrot labelled "tens of trillions", though I suspect this is an illusion in much the same way that Wikipedia is not valued at [peak cost of Encyclopaedia Britannica] * [global population with internet connection].

> And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing.

Yes but be careful anthropomorphising the LLMs too much. They're only somewhat human-like in their behaviour, and to the extent that they're human-like they demonstrate a huge range of personality disorders: https://www.personalitybenchmark.ai

Though plus side, apparently not evil: https://arxiv.org/html/2406.14703v2

I am not anthropomorphising the LLMs at all, I was talking about the obligations we put on humans using/making/etc. machines etc.

Hmm. I think I misunderstood what you meant by "experience policing people around financial incentives" in that case.

Simple examples here would be higher financial transparency obligations or more closely policing transactions.

> we need to focus on the issues at hand first rather than hypotheticals

This attitude is what's got us here in the first place, and if we continue thinking like this when we're going to go right over the cliff. The hypothetical cliff that's coming, but we've never gone over a cliff before so we keep on driving.

No. If OpenAI were being responsible and not criminally negligent, at the top of page 1 of the runbook would be "don't connect this to the actual Internet, even if the agent says Please."

That's what I assumed, but when I pasted this JS snippet from the blog post into the playground (ctrl+f "playground", selects JS) on the project page (https://davatron5000.github.io/microlighter/)

    if(document.querySelector('pre>code').length) {
         import('path/to/microlighter/microlighter.min.js');
    }

The highlighting looks completely different. I assumed the top-level commenter was being mildly obtuse, but it looks like they were (perhaps incidentally) correct