This fixation on "entire humanity" and "truly believe".
How about this, they think it's possible LLM could totally facilitate a mass murder of never seen before scale, but they continue to work on it because they're rich and sheltered enough they are not that worried that their family personally will be killed. The rest of humanity doesn't matter compared to making more money.
First don't get fixated on the "entire" humanity part. Even 10% should be unacceptable period.
Second you can't compare it to just one person's capability.
And it's never a doubt that really smart and insane people absolutely could kill a lot of humans considering modern technology. The reason they don't is because our society was historically built so that smart people don't want to or get stopped. For example ethics, religion, mental hospitals, self preservation instinct, police etc.
If anyone believes a sufficiently smart consortium of humans could exterminate 10% of humanity, that alone should be fixed by identifying the pathways through which it could be done. The existence of AI is not needed to be concerned about that possibility.
And dealing with those pathways individually would be far more productive than attempting to control the proliferation of algorithms that think, which in the long term is probably impossible. It also leaves us on far firmer scientific footing. An individual pathway - bioweapons, nuclear weapons, etc. - is far easier to reason about and accept/reject a notion of feasibility. Treating an ASI as a machine god and asserting futility in face of that god is not going to get anything done.
Part of the point in all this is that we can't predict what an ASI would do. All we can say for certain is that we would not be able to control it, because that follows a simple logic that between two entities with differing goals, the more intelligent entity is more likely to see their goals realized. It's interesting to me that you seem adamant that we identify and fix the pathways through which smart humans could exterminate 10% of humanity, but fail to see that preventing ASI is doing exactly that.
It's the AI doomers who have chosen to frame the argument in terms of extinction. Even if they were only claiming that 1% of the world population will be killed in the next 10 years, it would still be an extraordinary claim that hasn't been established with any amount of rigor. I'll reiterate the original point: these people are not serious, they're spreading alarmist nonsense, and they're undermining the public's trust in experts.
There is no proof possible. There is no experiment that can show something humanity scale will happen. There are even no remotely similar precedents in the past to extrapolate.
The actual established real evidence is that this tech is showing unprecedented capabilities (including destructive) and its safety guardrails are lacking.
Building for extreme scale constrained the possible architectural space. For a high level overview of the application architecture, refer to the seperate comment here in this thread.
It started initially by realizing that our app, Slyp, requires extremely distributed invoice and coupon processing as we started rolling out. So the architecture must handle such scale without issues.
This lead to persisting every incoming request (for commands) from the frontend and dismissing them only to be informed later of continuing with the status. This avoids processing requests when the system is overloaded with high traffic. So naturally the minimum is a log style ordered persistence. Initially kafka. Later our own Monolog built in rust which is substantially faster for our workloads and zero jvm and low memory and can run on servers and mobile alike.
Naturally the right point to consume this was into the command ingestion point in the Feature Architecture as noted in the other comment. This expanded the capability to all feature invocations including inter and intra service feature invocations.
This then progressively evolved into a much capable engine at which point we realized, this is a durable execution engine for command features but as a minor part of the whole Feature Architecture itself.
If we hold software accountable doesn't that mean software must have freedom? It doesn't make any sense.
reply