What kind of performance gains can you get from that? Seems to me that 99.9% of the compute happens remotely.

Probably minimal, it's mostly lower memory use from not having to boot a separate V8 engine with its own GC, like how Electron apps have to boot a separate browser engine. But CPU-wise it's not doing anything interesting on the client.

The neat thing for me is just not needing to setup a Node environment. You can copy the native binary and it should run as-is.

There are various solutions for turning node programs into standalone binaries, by more-or-less including Node within the binary. One I've used before with success is "pkg".

I would rather not have to package the entire runtime of a language just to run one program, hence Rust is a great choice for this.

Don’t have to wait 2 seconds for the runtime to start.

Yeah, they could probably get away with doing this in Go.

If you're parsing JSON or other serialization formats, I expect that can be nontrivial. Yes, it's dominated by the llm call, but serializing is pure CPU.

Also, ideally your lightweight client logic can run on a small device/server with bounded memory usage. If OpenAI spins up a server for each codex query, the size of that server matters (at scale/cost) so shaving off mb of overhead is worthwhile.

Probably uses less memory?

The big one is not having node as a dependency. Performance, extensibility, safety, yeah, don't really warrant a rewrite.

I get why OpenAI didn't invest time and money into this, but I do wonder if there's some reason that nobody have written a JavaScript frontend for LLVM.

There shouldn't be a reason why you couldn't and it would give you performance and zero dependency install.

What do you mean by “a JavaScript frontend for LLVM”?

Edit: ah, I see, I read “LLM” instead of LLVM at first! It's only after I posted my question that realized my mistake.

I'm not sure it makes sense to compile JavaScript natively, due to the very dynamic nature of the language, you'd end up with a very slow implementation (the JIT compilers make assumptions to optimize the code and fall back to the slow baseline when the assumptions are broken, but you can't do that with AoT).

> I'm not sure it makes sense to compile JavaScript natively, due to the very dynamic nature of the language

That's a good point, maybe TypeScript would be a better candidate.

TypeScript is just as dynamic as JavaScript.

For what it would take to compile TS to native code, check out AssemblyScript.

> but I do wonder if there's some reason that nobody have written a JavaScript frontend for LLVM

Astral folks are taking notes. (I wouldn't be surprised if they already have a super secret branch where they rewrite Python and make it 100x faster, but without AI bullshit like Mojo).