1. Upgrade your JDK for the best performance (as the article says, the slowdown is gone in JDK 26).
2. Don't try to help the GC by pooling objects. Mutating old objects can be expensive, while allocating new ones is cheap (at least for objects that don't do some exceptionally expensive initialisation).
Object pooling still has its place, but like any optimization it needs to be based on benchmarks and shouldn't be done haphazardly. Blindly pooling objects will lead to regressions and resource contention more often than improvements.
There are also middle ground options, like pooling objects but giving the pool a lifecycle that is tied to a request.
The problem is that 1. it's not easy to beat the JDK's GCs at memory management (assuming you've picked the right GC for your workload) especially as they keep getting better and better, and 2. how a pool behaves relative to the GC depends greatly on the GC algorithm (e.g. the same pool could help a bit with, say, Parallel GC, and hurt significantly with G1 or ZGC), and the different GC algorithms also tend to change significantly from release to release, so it's hard to write a good pool that can remain good both across different GCs and across different runtime versions.
In particular, the JDK's GCs are heavily optimised for short-lived objects with high allocation rates. What you want to avoid is temporary data finding itself in the old gen. The newer GCs may dynamically size the young generation to match your program's natural meaning of "short lived", but the longer an object lives, the higher the chances it ends up in the old gen. You want only objects that stick around for a very long time (and have a low allocation rate) to end up in old gen. If you just write naive code, chances are things will work out well. Once you start being clever, you're taking a risk.
So if you're willing to profile your program, with a workload that's representative of production workload (a microbenchmark is useless) on every runtime version and potentially change your "manual optimisation" every six months, you can try. But if not, the advice for the best performance over time is to rely on the platform and let it do its thing. The reason is that the JVM is continuously being optimised for "normal programs". If you're doing anything too clever, you may find that in a future release, your code is making things worse because the optimisations that target normal code don't help your code (or could even treat it as unusual and have it hit slow paths).
I once spoke to a company that were very proud in getting something like a 10% improvement over naive code in Java 8, thanks to some hand optimisation they worked a lot on, only to discover that it caused a 15% regression compared to doing nothing special on JDK 11.
Where pooling does sometimes win is for stuff like intermediate buffers for compression or decompression, since a Java alloc will zero the memory, which for sufficiently large buffers is much costlier than the allocation itself, and in such a case you don't care if it's zeroed.
Removing allocation pressure can also have effects on other parts of the system, but that is anything but trivial to measure or reason about.
Honestly, I don't really understand why G1 is being pushed so hard.
The parallel collector is a perfectly fine collector, particularly for smaller heaps. Even the serial collector isn't bad for things like a containerized environment, yet G1 replaces it by default now [1].
It's not a bad algorithm, but especially when you start talking about sub 2G environments I've not seen a situation where the parallel and serial collectors won't handily beat G1 on pretty much every metric. Major collectors with modern CPUs just doesn't take much time for a lot of memory.
If anything, I think it's not unlikely that ZGC will become the default at some point, as it matures. It's hard to beat Parallel on batch workloads, although G1 is getting there. ZGC is unparalleled for low-latency (GC pauses are just gone). G1 is intended to offer a compromise that could be a reasonable default.
I have no qualms with ZGC being the default. The low latency that it offers at near G1 speeds is a very good trade off (IMO).
I just have a problem with G1 because in my experience, the best place for it is fairly large heaps. Get something sub 2 or even 10G, especially if you have a few cores to offer, and the parallel and often even the serial collector will give G1 latency even on major collections with superior throughput and overhead.
I would be very surprised if ZGC became the default, because it incurs a significant overhead penalty to eliminate those GC pauses. All else equal you're effectively just sacrificing throughput for latency, since it's doing a bunch of extra housekeeping in the background (foreshadowing...) That's a perfectly reasonable tradeoff to make if low latency is a priority (or perhaps more importantly if having very consisent/predictable latency is a priority) but in most Java projects I've been exposed to that's been a tertiary concern at best. Frankly, I question if most Java developers are even aware that they're allocating physical memory when they type 'new'...
In the modern enterprise Java world (that I've been exposed to) it's very common to have a mandate that all components deploy a minimum of N instances across X regions for resiliency. By design that almost always means you're deploying at least 2x more compute than you strictly need, so the top priority is generally minimizing per-instance overhead to minimize cloud spend.
For example, the default templates at my current company deploy something like 0.25-0.5 vCPU per instance, and therin lies the rub. ZGC performance is _catastrophically_ bad with <=1 cpus because when there's only one core, any "concurrent" GC events become full on stop the world events. We had someone pilot a change to the default JVM args for all components because they heard that ZGC would reduce latency, only to discover that basically all of our microservices immediately failed their perf tests. For the first one I spot checked, throughput was down ~90% and p95 went from ~40ms to >1s, because more time was being spent on "background" GC than actually servicing requests.
Hope that didn't come off adversarial. I just find GC fascinating, and ended up spending a bunch of time working with the team that owns those defaults to draft general recommendations. TLDR is that when in doubt don't specify/let the JVM pick for you, and don't be surprised if it picks serial :)
> I would be very surprised if ZGC became the default, because it incurs a significant overhead penalty to eliminate those GC pauses
That throughput penalty is not very high with generational ZGC. It's not zero, but it's not very high, either. What ZGC mostly does is spread the memory management activity more evenly across the duration of the program (this does have a cost due to barriers being active more, but it's not huge). But we have some work planned to improve ZGC even further, which is why I didn't say I think it will become the default imminently, only eventually.
> ZGC performance is _catastrophically_ bad with <=1 cpus
That may well be true, but the JVM can automatically choose a different default algorithm for these circumstances. Indeed, until very recently, the default for low-CPU environments was different (Serial) than for bigger ones (G1).
I am happy to see JDK versions actually becoming faster and lighter over time. Nice contrast with other platforms that seem to be moving in the opposite direction.
I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language that offers safety.
So right now, when we want control, we need to give up some safety, but weaker things are still helpful.
Also, in low-level code, the problem of "I might forget to do something" sometimes clashes with the problem of "I need to see exactly what operations are done and where". Various kinds of implicitness help with the former at the expense of the latter.
I'm not saying this is universally better than other approaches, but many people who do serious low-level programming would prefer this.
> With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language that offers safety.
This is a very, very, very common claim. And unfortunately I have no other way to describe it other than a strawman.
In 95% (at least) of the application that need systems programming (not to talk about all applications that don't necessarily need it but will benefit from the performance and it wasn't an option because C++ wasn't an option), you have at most 20% (wildly overestimating) of code that needs to be unsafe. The rest could be completely safe. And amongst code that must be unsafe, you can very commonly encapsulate it in some safe pattern. Many times even extract it to a reusable crate.
> you have at most 20% (wildly overestimating) of code that needs to be unsafe
Obviously, but that doesn't help me if the complexity in the unsafe parts is made worse, while the safety helps the parts where little help is needed. It's not like the danger in a C program is spread evenly, either.
> but will benefit from the performance
Not so much. Safe Rust is faster than Python and Go for sure, but is, on average, about as fast as Java and C#; sometimes faster, sometimes slower.
So, steady-state performance that's about as good as Java or C# on average, but with memory safety, much smaller baseline executable size (yes, even compared to GraalVM Native Image; I haven't checked current .NET AOT), faster startup (yes, I know that's what Native Image does optimize), and lower memory footprint? I'll take that deal, even if there's a substantial gap between safe Rust and C++ or Zig. I badly wanted something like safe Rust when working on desktop applications throughout the 2000s and into the 2010s, and now it's here, with a strong and growing library ecosystem.
If that were the actual tradeoff, I'd take it, too (and BTW, Java's memory safety is much better than Rust's, but that's beside the point now). Remember that even 25 years ago you had a very similar thing with C++ vs Java, aside from memory safety, but people didn't make an exceptionally big deal about it then. The main problem with C++ was that, over time, it gets harder and harder to evolve the program, especially while keeping performance reasonable (you can make C++ programs easier to evolve by using a lot of dynamic dispatch and the refcounting GC, but in low-level languages you pay for those in performance dearly).
So what you're really getting is, typically, a smaller executable, a faster startup, and lower footprint (which is actually a much more complicated matter, but I won't get into it now) in exchange for significantly higher evolution and maintenance costs forever. This is a good and reasonable tradeoff for small programs, especially CLI tools, and not a very good tradeoff for larger and/or longer-lived programs.
> Obviously, but that doesn't help me if the complexity in the unsafe parts is made worse, while the safety helps the parts where little help is needed. It's not like the danger in a C program is spread evenly, either.
The complexity is made worse for specific, isolated, encapsulated and reusable code, while all other code becomes significantly safer? That's a deal I'll take at any time. And again, empirical evidence proves that to work.
> Not so much. Safe Rust is faster than Python and Go for sure, but is, on average, about as fast as Java and C#; sometimes faster, sometimes slower.
Nonsense. In all benchmarks I saw Rust is significantly faster than C# and Java, sometimes up to 2x-3x, and about on par with C++ (can be a few percents slower but that depends on many things). In fact Go is closer most of the time.
> The complexity is made worse for specific, isolated, encapsulated and reusable code, while all other code becomes significantly safer? That's a deal I'll take at any time.
That's not the deal I'm getting on either side of this.
> In all benchmarks I saw
If you trust those benchmarks then you deserve whatever you pick. I was talking about experienced experts who understand performance. Low-level languages can help your performance when the program is small and they generally hurt it when it grows large, evolves through many people etc. This is something that people with a lot of experience in low-level languages know.
> If you trust those benchmarks then you deserve whatever you pick. I was talking about experienced experts who understand performance. Low-level languages can help your performance when the program is small and they generally hurt it when it grows large, evolves through many people etc. This is something that people with a lot of experience in low-level languages know.
Appeal to (an unnamed) authority? I consider myself an experienced experts who understands performance and this also matches my experience. While you often can reach the same level of performance in Java or C#, it involves horribly unidiomatic code, unlike in Rust (or C++).
Well, if you have a couple of decades of experience with low-level programming, you know that AOT compilation and non-moving pointers carry intrinsic runtime overheads that manifest as programs grow large and complex, and run for a long time. It's these very overheads that moving collectors and JIT compilers are designed to reduce, and it's also the very thing benchmarks don't measure. A TCMalloc runtime is almost the same size as Java's most sophisticated GC, and it still can't keep up because of the fundamental overheads. In low-level programming we try to avoid these overheads by avoiding dynamic dispatch and dynamic heap memory, but it gets harder as the program evolves.
It's true that even in such programs you could, in principle, reach the same level of performance of Java in C++, but in practice it's very, very hard. This is why most large and long-running programs have abandoned low-level programming languages. It's easy to get excellent performance when the code is small, regular, and new, but over time it gets harder and harder.
In general, low-level programming languages yield relatively fast small programs, but relatively slow large programs, and with Java/C# it's generally the opposite. The low-level control that helps performance when you're small, starts hurting it when you're big.
> This is why most large and long-running programs have abandoned low-level programming languages.
That's not true, as evidenced by the fact that this move has started before extremely sophisticated JIT compilers or garbage collectors were invented. The reason was not because managed languages were faster or even had equal speed, but because of the costs associated with memory unsafety (not just security), exactly what Rust prevents (which was of course not available then).
> But after a few weeks, it compiled and the results surprised us. The code was 10x faster than our carefully tuned Kotlin implementation – despite no attempt to make it faster. To put this in perspective, we had spent years incrementally improving the Kotlin version from 2,000 to 3,000 transactions per second (TPS). The Rust version, written by Java developers who were new to the language, clocked 30,000 TPS.
You also ignore the impact of memory usage, where unmanaged language have an even greater edge (yes I know it is possible to optimize managed languages' memory consumption as well. Not to the same amount and often at the expense of speed).
> That's not true, as evidenced by the fact that this move has started before extremely sophisticated JIT compilers or garbage collectors were invented.
I don't know how long you've been programming, but that's not true. In the late nineties and early aughts I was working on large, performance-critical, soft- and hard-realtime systems, and we only started moving away from C++ when Java started beating its performance.
> The reason was not because managed languages were faster or even had equal speed, but because of the costs associated with memory unsafety (not just security), exactly what Rust prevents (which was of course not available then).
That's a myth, and a fairly recent one. Sure, there were non-performance-sensitive programs written in slow languages for a long time. But the industry was mostly using C++ for anything that needed to be big and fast, and back then "memory safety" was mostly just another type of bug. It was nowhere near reason enough to use slow languages, which is why we didn't use them.
Lack of memory safety is a serious problem, but the claim that it's the biggest issue with C++, let alone the one that's always been considered the biggest issue, is just a myth. Back then it was certainly considered no bigger an issue than the language complexity, compilation time, and even performance issues in large, long-running programs.
> You can see empirical evidence of this, for example, by the post about Aurora DSQL rewrite in Rust
I talk to the people at AWS, and this is not the evidence you think it is. First, their problem was primarily with GC pauses, and it was before pauseless GCs. Second, the codebase isn't very big. Third, because Java and C++/Rust offer similar performance - sometimes one wins, sometimes another - you expect to see exactly that. I can tell you that we recently wrote a distributed cache in both Java and Rust simultaneously (using the pauseless GC). The Java version achieved twice the throughput of the Rust version, and significanly better latency across all percentiles. So sure, on the smaller end, there are programs where Rust would be 2x as fast as Java, there are programs where Java would be 2x as fast as Rust, and on average they're about the same. But over time, Java's advantage starts to show as it makes it easier to keep the good performance over years of evolution.
I know that. I also know that JITs cannot optimize to the same amount as LLVM due to the time limit, and that C++ and Rust are allocating much, much less than Java and even C# or Go, so a faster allocation scheme is much less needed there. I'm not saying that faster allocation or fragmentation cannot yield gains for some specific programs, but even in those cases it's usually possible to alleviate the costs with wise organization of allocations (including using arenas etc. in some places), and they're also offloaded from the better-optimizing compiler.
> I know that. I also know that JITs cannot optimize to the same amount as LLVM due to the time limit
Yeah, this is not true, and there's no time limit. I mean, maybe some JIT compilers, like JavaScript's have a time limit, but their goal is to run JS at an acceptable speed. Java's JIT is intended to reduce the runtime overheads of AOT compilers, and the only way to do that is by optimising significantly more than AOT compilers, obviously not less (otherwise, we'd just always use an AOT compiler).
You can easily see why there's no time limit if you understood how Java's optimising JIT works. First, code is run in the interpreter and some profiles are collected, then a non-optimising JIT runs and continues to collect profile, and finally the optimising JIT runs. The vast majority of the time is spent waiting for profiles to collect, and so if compilation itself runs, say, even 3x slower, it won't even be perceptible. Also, because we have profiles, we don't have to compile much of the program at all, because we know what the hot spots are. Initialisation code that runs once is never compiled (remember, the focus is long-running programs, exactly those that low-level languages have trouble with).
Finally, the reason sophisticated JIT compilers can optimise more - which is why they're used in the first place - is thanks to speculative optimisation. AOT compilers need to spend a lot of time on optimisation, and even then they are limited, because they need to prove that the program transformation is valid (i.e. that there's no miscompilation). The power of JIT compilers is that they don't. They only need to speculate that a certain profile will continue to be in effect. So if so far some virtual call always hits a certain target, they can go ahead and inline it (not only to the cost of a regular call, but to no call at all, and then they optimise the whole inlined code). If they're wrong, a fault triggers and they decompile the relevant subroutine going back to the interpreter and non-optimising compiler.
> and that C++ and Rust are allocating much, much less than Java and even C# or Go, so a faster allocation scheme is much less needed there
This is true, but the causaility here is that the reason we avoid allocation in C++ is precisely because it's so slow.
> but even in those cases it's usually possible to alleviate the costs with wise organization of allocations
The problem is that this is true in principle. In practice this is certainly true in smaller programs. In larger programs, this work is not easy at all, and you find yourself doing harder and harder work just to keep up.
> including using arenas etc. in some places
One of the reasons I'm excited about Zig (I'm a low-level programmer) is that it makes arenas much more viable. Arenas in C++ and especially Rust are not really a pleasure to work with, and they're viral and a constant maintenance burden. BTW, the reason moving GCs are so fast is that they work quite similarly to arenas.
> and they're also offloaded from the better-optimizing compiler.
It's a worse-optimising compiler. In C++, I use templates to achieve similar optimisation to what Java does, and in Zig I can use comptime, and again, it's certainly possible but it's hard work. You can't let the templates explode all over the codebase, and, as it evolves, you have to go back and profile and take out the ones that no longer help, replacing them with new ones.
Just to tell you a bit about me, I was a C++ programmer for many years, and when Java showed up, like many, I was sceptical. When I saw that the JIT + moving collector hypothesis actually accomplishes its goal in reducing the overheads we were seeing in C++ in many situations, I went to work on the JVM. Back then there were still latency tradeoffs due to GC pauses, but GC pauses no longer exist as of three years ago.
Now, a lot of people, including some of the world's top compiler and memory management experts, believe that the vision of using JITs and moving collectors to address the performance problem of low-level languages is working exceedingly well. We can certainly argue about under which conditions Java wins and under which C++ (or Zig it Rust etc.) win and how common they are, but people who think low-level languages win across the board or almost across the board clearly don't know what's going on. Early on it was people who were sceptical about how effectively JITs and moving collectors could do their job in practice (even though the theory was clear), but these days I think it's mostly people who haven't struggled with performance issues in low-level languages long enough, and just see that for small or young programs they work fine. They always were. Writing a new program in C++ was never harder than writing a new program in Java, and the performance was great (and people weren't concerned about memory safety in particular). The problems came later - in the 5th year, the 10th year, etc.., when the cost of evolution and trying to keep performance good were piling up.
In addition to that, even unsafe Rust is not like C. It disables a limited set of checks, but Rust's type system, ownership model and bounds checks remain in force.
Not to mention the availability of advanced tooling like MIRI.
Yes you need to vet touching safe code. Which is why you keep things private, encapsulate them, and extract them into reusable crates.
The most important reason unsafe code is harder to write than C or C++ is that you must keep soundness, something none of these languages have. But yes the different rules also play part (although: do you know a single C or C++ codebase that does not violate TBAA? Some just disable it in the compiler, making them non-standard, while some just leave it potentially exploitable).
But the most important answer is the empirical evidence like I brought above. We have empirical evidence C and C++ codebases cannot be secure. We have empirical evidence Rust codebases can, even with unsafe code. Therefore, Rust is safer, period.
> Do Rust libraries, including std, historically have had UB bugs?
Did C or C++ libraries, historically, have UB bugs? Sorry, that just amplifies the strawman.
> Can Miri catch everything?
Miri is a dynamic analyzer, aka. a sanitizer. It will catch anything you test. It's like in C and C++, except you only need it for unsafe code.
> Are all the rules of unsafe, pinning, etc. fully specified and easy to learn and reason about?
Fully specified? People are working on it (are C's and C++'s UB rules fully specified? I'll save you the answer: no. Yes there is a standard and it's woefully incomplete).
Easy to learn and reason about? Probably not, which is why not everyone should be writing unsafe code.
Possible to learn and reason about? Absolutely yes. Especially with existing and emerging dynamic and static analyzers.
The claim isn't "there's no unsafe". You've linked one file out of an entire stdlib; it uses unsafe to implement its algorithm, and of all the Rust code that could exist, this has one of the highest requirements for being maximally performant.
Now if you'd said "most of the Rust std library is unsafe", or "most Rust code is unsafe, you'd have a good rebuttal. But that's not the case.
> And, if you have an unsafe block that is 100% correct, but it relies on safe code being correct, do you need to vet all that safe code? Potentially whole modules needing to be vetted?
Then the unsafe block is not 100% correct. I can slap a wrapper around memcpy and call it "safe", and say that if anyone passes wrong parameters it's their fault. Rust as a language says I'm at fault for saying it's safe though.
> Is unsafe Rust code generally harder to get correct than code in other languages, due to...
Harder than other systems programming languages? Having worked in a fair few, I disagree. Harder than "higher" level languages? Some of them yes, some of them no; I've seen "simple" languages admit very poor architectures, and fall in a "safe" heap when the project has to grow.
I've written systems level code (drivers and os code) for years and outside of ffi, I've managed to go on year long stretches without touching unsafe. It's really not a commonly needed tool in a well architected code base with good libraries to encapsulate common reasons it might otherwise be necessary. And we don't really consider using unsafe taboo, it's just not necessary.
But the point of unsafe {} in Rust is not that you should never use it, it's that it creates a clear boundary between code that is safe and the code that needs that lower level control. In other languages, everything is inside an unsafe block. If everything you do requires such low level control over every allocation and access, it sounds like you should be using assembly.
While I think this is a good idea, I think this the importance is massively exaggerated. Not everything in other languages is unsafe, there are also different features one can distinguish where some are safe and some are not. It is not as clearly labelled and a set of features one must screen for instead of one keyword, but pretending this then makes it everything unsafe is disingenuous. At the same time, the distinction is Rust is also not always that clear, as the correctness of the unsafe block may depend on logic outside of the block while soundness of the safe parts may be compromised by issues in the unsafe blocks.
Objects belonging to multiple double-linked lists at the same time. Easily done with intrusive lists. Safe rust would require Rc/Arc: penalty both on memory usage and cpu time.
That's not "for instance", that's literally the only place Rust has unfixable UB on embedded (code on OS has other such things, e.g. reading/writing to `/proc/self/mem`).
> projects that need performance often use unsafe
You'll be surprised to hear how often it's not needed at all. And when it is, you'll be surprised to hear how many times you can still avoid it with some tricks. Contrary to popular belief, performance isn't the most common reason for unsafe (FFI probably is).
I haven't needed `unsafe` for performance since crates like zerocopy etc exist. It's been years, and I've worked hard to shave nanoseconds off of code, using valgrind to measure single digit changes to branch predictions.
Except the point that Zig should do better than Object Pascal, Modula-2, with solutions already available on Insure++ and friends for use after free, 30 years ago.
First, it's not a C++ feature. In C++ you tell the compiler how to lay out objects in memory. Here you declare what properties your class has (e.g. whether it needs identity or not), and the compiler decides how to lay out each of its instances in memory, and may automatically do it in different ways in different places. So the principle that "you tell us what semantics you want and let the compiler figure out the implementation" remains in effect.
Second, the reason why the compiler cannot infer on its own that a class does or does not need identity without you declaring it is that the use of identity can be in a different module.
The problem is that the people getting good results with AI-assisted formal methods are the same people who get good results with formal methods without AI assistance. They then extrapolate the benefits they are getting from AI today to what it may do for others in the future, and this is where we get into trouble.
There's a lot of art to using formal methods around how to specify the system at the right level of abstraction (to make verification tractable) and how to specify the correctness properties so they can be easily evaluated. Even with AI assistance as it currently exists, users need to know formal methods well enough to at least understand the specification of the system and the correctness properties, which requires ~90% of the effort of learning formal methods in the world before AI.
But the real hope is that one day AI will be able to use formal methods correctly on its own, benefitting those who don't know formal methods. AI can sometimes do that today, but sometimes isn't good enough for people who don't know formal methods. It is certainly possible that soon enough AI will be able to do this more reliably, but then we get into the hard problem of speculating the "AI future". It is very hard to predict what an AI that can take over the art of using formal methods cannot do. Predicting that AI will be able to do that yet not be able to collect requirements and build software autonomously, or even come up with the idea for what software to build in the first place, or even replace the software's users seems arbitrary to me. In other words, if people think AI will take care of the verification letting us focus on requirement validation, my question would be, why wouldn't an AI that knows how to verify also know how to validate the requirements? For that matter, why wouldn't it also know how to replace the users altogether?
1. The study of complexity classes isn't intended to dissuade people from writing certain programs. It's intended to understand the nature and theoretical limits of computation. As far as practice goes, it can be used to show where heuristics are needed. Saying it's overrated is like saying calculus is overrated because most people don't need to use it every day. And BTW, many important problems are in classes believed to be way harder than NP (i.e. NP-complete is the easiest of the hard famous complexity classes). E.g., I've seen some people brag about some configuration language being easy to mechanically analyse because it's not Turing-complete, while in fact it's at least PSPACE-hard to analyse.
2. When there's some large set of instances of some NP-hard problem that are tractably solvable in practice (like SAT), the importance of that is that there's some non-NP-hard subset here. Indeed, SAT is FPT (fixed parameter tractable [1]), an "easier" type of NP, for which decomposition can help. In contrast, graph colouring is thought to not be FPT.
> Saying it's overrated is like saying calculus is overrated because most people don't need to use it every day.
You should stop thinking by analogy.
The article was showing the difference between mathematicians and engineers. For the mathematicians that created Computation Science, the only interesting solutions are complete solutions to general questions, whereas for engineers it's perfectly acceptable to eliminate some corner cases, thereby solving a reduced and simplified version of the general problem.
> For the mathematicians that created Computation Science, the only interesting solutions are complete solutions to general questions
Except that's not really true, which is the whole point of the finer computational classes. If many instances are far from the worst case, that tells you something interesting about the class, which is why we have things like parameterised complexity. People who think that the theory is only interested in the general case of the broad classes you learn as an undergrad are just not sufficiently familiar with the theory.
I did study a bit of complexity theory back in the day, and it does seem a bit theoretical, but not so long ago, I ran into an very prominent manifestation. To help someone, I had created a simple web page which also had an input field for the css that was applied directly to the page. To avoid flickering and weird effects because of malformed css, I wrote a regexp to check the CSS code. It's a very simple language, after all, and it didn't have to be perfect. It worked well, until at one point, a simple typo locked up Chrome for 1 minute 10 seconds. Parse time just jumped from less than 100ms to 70000ms by adding a single character. That's when you feel what exponential means.
I just want to point out that complexity theory isn't really about analysing the complexity of a particular algorithm (that's the subject of algorithm and data structure analysis, where students learn big-O notation), but about understanding the complexity of a problem independently of any algorithm (i.e. what problems couldn't be tractably solved by any algorithm).
> The article was showing the difference between mathematicians and engineers.
No. Many engineers AND mathematicians worked for a long time to get us to a stage where Amazon can solve a billion SMT problems a day. To contribute, all of them had to understand the theory this article calls overrated.
Some mathematicians certainly did, but there's a very large undercurrent in CS, as well as Mathematics more in general, of utter disinterest for applications as well as the idea that the more general a solution, the more "worthy" it is. That was really obvious from the words of the professor cited in the article.
I would to say I support that view. What is the purpose of modern science if not to discover truths you can apply universally? If you state: ‘this particular apple falls to the ground’ thats not a scientific discovery, there must be some general applicability.
2. That the worst-case is very hard usually tells you that many instances will be hard (unless you discover an easy subclass), as is the case here. And when many "natural" instances are easy, that means that the problem is more interesting than perhaps previously thought, and it requires and receives more research, not less. If most instances are near the worst case, it means you know all there is to know about the problem; when they're not, it means there's more to study.
A program that enumerates all theorems in ZFC and stops when it proves a contradiction (e.g. true = false). Encoding a program that is equivalent to that directly as a Turing Machine in merely 748 states: https://www.scottaaronson.com/papers/bb.pdf (meaning that we cannot prove an upper bound on the 748th Busy-Beaver number, but there are probably even smaller undecidable TMs).
But my favourite example (shown here in Java) demonstrates the difficulty of analysing simple, realistic programs without necessarily being undecidable:
long foo(long x) {
if (x <= 2 || (x & 1) != 0)
return 0;
for (var i = x; i > 0; i--)
if (bar(i) && bar(x - i))
return i;
throw new Error();
}
boolean bar(long x) {
for (var i = x - 1; i >= 2; i--)
for (var s = x; s >= 0; s -= i)
if (s == 0)
return false;
return true;
}
Even in this case where even the input space is finite (and so everything here is definitely decidable), we simply don't yet know whether there is some x for which foo(x) throws, let alone if we made the input unbounded by using BigInteger instead of long.
You have clearly not encountered theoretical computer scientists. They love to create all kinds of complexity classes and theorems to capture things like heuristics and approximation algorithms and other things which work in practice but not theory.
In fact that’s a big research thrust right now, to understand why many real-world SAT instances are solvable quickly while others are not, and where the threshold between them lies
I studied mathematics, and can attest that the attitude of the professor mentioned in the article is very representative of an older generation of mathematicians. Maybe the younger ones are different.
I also studied mathematics, and I can tell you that this was not my experience.
Rather, if a problem is NP-complete/NP-hard it means that we cannot expect a general fast algorithms for exactly this problem (in other words: more mathematics is required, which mathematicians of course love).
But it is absolutely known that there exist other strategies:
- Develop algorithms that work well in practice and make understanding why they work so well in practice your career.
- Find out whether there exists something that makes the instances that occur in practice different from those instances that were used in the proof that the problem is NP-complete/-hard.
- For optimization problems: develop some fast algorithm which guarantees some approximation factor.
It't not a matter of generations. There are plenty of old mathematicians who are very interested in applications and plenty of young mathematicians who are only interested in generalized abstract nonsense. It's more of a difference in personality that will always be there.
> E.g., I've seen some people brag about some configuration language being easy to mechanically analyse because it's not Turing-complete, while in fact it's at least PSPACE-hard to analyse.
Isn't even just the question of minimising the length of a regular expression PSPACE-hard or so?
> E.g., I've seen some people brag about some configuration language being easy to mechanically analyse because it's not Turing-complete, while in fact it's at least PSPACE-hard to analyse.
I don't get your point here. What analysis are you talking about?
I believe the claim usually made about non-turing-complete languages is that it is possible to prove specific properties with little to no calculations, that would be otherwise hard to calculate. For instance, the time needed to determine that an Idris program will eventually stop is litteraly 0 seconds.
> I don't get your point here. What analysis are you talking about?
Determining any kind of non-trivial property (i.e. a property that isn't true for all or none of the programs in the language).
> I believe the claim usually made about non-turing-complete languages is that it is possible to prove specific properties with little to no calculations, that would be otherwise hard to calculate. For instance, the time needed to determine that an Idris program will eventually stop is litteraly 0 seconds.
It's not the non-Turing-completeness that makes that practical. Let's take your example of Idris:
1. If a program's termination is hard to determine, then it will be hard to write it in Idris. I.e., the effort isn't gone, it's just shifted elsewhere. And if the program is easy to write in Idris, then its termination is also easy to prove in other languages (Idris effectively requires you to write a proof of termination, but you can write the proof for any language).
2. The importance of this is not as high as you think. For example, we can trivially rewrite all of the world's software in an always-terminating language (so not-Turing-complete), by changing the semantics of all programs to terminate after 2^100 steps. This will not affect the behaviour of any software, and you can see why it also won't make determining any of their properties of interest any easier.
So yes, Idris makes termination a trivial property for Idris programs, but it doesn't make the effort of determining whether an algorithm terminates or not easier (you just have to do it _while_ you're writing the program instead of after), and it doesn't, by itself, make any other property (which remains non-trivial) easier, such as "does the program terminate in fewer than 2^100 steps?"
> That is the benefit of non-turing complete languages, though.
That's simply untrue. You can write arbitrary proofs about programs in Turing-complete languages, too. In fact, most formal proofs are of programs written in Turing-complete languages.
> but languages like Rust or Idris make it simple because they force correctness early in the coding process.
Rust doesn't actually let you do that, though. In terms of the expressive power of proof, it is far closer to C than to Idris (in fact, from Idris's vantage point, Rust is almost indistinguishable from C).
> 2. When there's some large set of instances of some NP-hard problem that are tractably solvable in practice (like SAT), the importance of that is that there's some non-NP-hard subset here. Indeed, SAT is FPT (fixed parameter tractable [1]), an "easier" type of NP, for which decomposition can help. In contrast, graph colouring is thought to not be FPT.
Sorry but I need to clarify here. "SAT is FPT" does not mean anything. FPT only makes sense when you tell what is the *parameter*. Every problem is FPT when parametrized by the input size so graph colouring and SAT are FPT wrt to the size of the input (the graph and the formula respectively). What you meant: graph colouring parametrized by the number of colours is unlikely to be FPT (since it is W[1]-hard). SAT is FPT for many parameters such as treewidth (of the formula). Oh, and btw, graph colouring is also FPT when parametrized by treewidth (of the graph).
I take your correction re graph colouring, but the rest is pointlessly pedantic (especially "FPT in the length", which is trivial and so typically excluded from the definition).
Now, I don't know if SAT being FPT (in a parameter of interest) has anything to do with the surprising ease of many "natural" instances (even with many variables), but my point was that when many instances are easy, that doesn't mean that the class is irrelevant, it just means it's more interesting (as there's obviously a tractable subclass, albeit one we haven't yet defined succinctly).
Late answer but I need to be pointlessly pedantic again, because it is apparently what it is called to correct something that does not make sense. That said, I agree with your way of reformulating your point.
To first address the pedanticness: "FPT in the length of the input" is not excluded from the definition of FPT even if it is not an interesting case. Excluding these cases from the definition would make the theory uselessly complicated.
Now, back to the point of my first comment. It was mostly to show that you cannot use "FPTness" as an argument to justify that SAT is "easier" than graph colouring. I used "length of the input" to give an easy counter example, to show that FPT is not "an easier type of NP". It never has been. It is a way of understanding and measuring the complexity of the problem finely, to isolate hard parts of the input from the rest. What the W[1]-hardness of graph colouring parametrised by the number of colours tells you is that this parameter is not a relevant parameter and that's it. Graph colouring is FPT for many relevant graph parameters such as treewidth, clique-width (SAT is not even FPT wrt clique-width, but again, you cannot really use the concept of FPT to compare problems).
You will have a hard time using theory to justify that SAT is easier than any other NP-complete problem, because from the theory point of view, it is the hardest NP-problem you can get. Unless something unexpected happens in complexity theory, SAT cannot be efficiently solved in randomized time, SAT cannot be solved in sub-exponential time, SAT is complete under parsimonious reductions so you can basically take any NP-complete problem and build a CNF formula whose models are isomorphic to the solutions of the original instance etc.
The success of SAT solvers does not mean that SAT is easier than other NP-hard problem. It means that many combinatorial problems we need to solve in practice are "simple enough" that using the incredibly optimized smart way of bruteforcing the solution with a CDCL SAT solver is good enough. Now, take a cryptographic instance, translate it into a CNF formula and call a SAT solver, I doubt it will shine. Despite many attempts, this behaviour has never clearly be explained by the fact that the complexity of CDCL is FPT wrt a parameter that is small in industrial instances.
> You will have a hard time using theory to justify that SAT is easier than any other NP-complete problem
"Easier" perhaps isn't the right word (in terms of reducibility, it certainly isn't), but FPT is just a way to demonstrate that NP-complete problems can be different from each other in ways that matter (with respect to "natural" instances).
> Now, take a cryptographic instance, translate it into a CNF formula and call a SAT solver, I doubt it will shine.
Exactly. And researchers are very much interested in these differences that arise in natural instances and are hidden by "crude" reduction.
> It's intended to understand the nature and theoretical limits of computation.
Not in a general sense, at least for standard complexity theory. It only deals with a very specific model of computation. Anyone with a sufficiently solid grasp of metamathematics intuitively understands that the distinction between solve and verify is nothing but a description of how badly matched our foundations are for the structure we're trying to view.
... This is the second time today I've posted about foundations like this.
Standard complexity theory focuses on answering questions when our substrate behaves like a Turing machine with multiple tapes.
Consider it like this, if the answer is in our system's axioms, we don't have to do anything. In a trivial sense that means we're just given the answer table, but it's also true if our substrate matches the model of computation its simulating. IE for an SLD-Resolution machine, running an SLD-Resolution object language, unification is worst case O(1). This is a degenerate case of course, but it's an example of something that's not realizable on a Turing machine's semantics where the worst case is in... EXPTIME? It's not great.
The more we treat our substrate like building blocks, and less like a holistic oracle, that changes our complexity landscape. Complexity theory was never about studying that whole landscape.
You might want to say CT is pragmatic and focused on realizable machines. There are two problems with that:
1. There's nothing special with the baseline used for complexity theory other than its familiarity. Reality is our ultimate substrate. The universe is not Turing tape. There is absolutely no serious basis upon which an argument against substrates can be made, especially with how little we know and understand about the universe.
2. Complexity theory isn't so pragmatic to only study the finitely bounded, which also changes everything. There seems a very tight upper bound on information in the universe. Even studying up to it as a limit is decidedly not pragmatic in the slightest. This is perfectly fine of course, the problem only enters in when we want to be "pragmatic" on some things, but not others.
I also want to clarify: There are higher orders of complexity theory that have generalized a lot of its concepts, even into hypercomputation which is cool, but then there's another problem I didn't mention. Complexity theory still isn't about what he said. It quantifies that distance between prove and verify, but it doesn't study the set of all those distances and how they arise. It just quantifies them one at a time and has only a limited number of things to say beyond that. What he described is simply mathematical logic.
We actually understand quite a bit about the universe and the kinds of computers we can build. People also think about computers in speculative physics scenarios, eg closed timelike curves can be used to solve pspace complete problems.
I don’t think you can plausibly argue that complexity theory‘s base assumptions are a bad choice, at least not in the sense that you would assume that you can build exponentially more powerful computers in the physical universe. In fact, concerns about energy densities, limited amounts of matter, and the speed of light make it more difficult than typical machine models assume.
> Standard complexity theory focuses on answering questions when our substrate behaves like a Turing machine with multiple tapes.
This is not true. Complexity theory very much looks at complexity under different models (alphabet size, oracles, circuits). It's just that often (e.g. in the case of alphabets), there is a reduction of known complexity between two models.
> It quantifies that distance between prove and verify, but it doesn't study the set of all those distances and how they arise.
> there is a reduction of known complexity between two models.
Every single time I've seen, for example, the lambda calculus be assigned cost semantics, it usually looks like what you would expect out of a Turing machine's simulation of it. Often times, they're explicit about it: https://www.sciencedirect.com/science/article/pii/S030439750...
For me, I can't accept that this is the criteria of "reasonable." Especially not for abstract theory.
I did try to indicate I'm mostly talking about standard complexity theory, the stuff you'd encounter on the surface level of the field. I'm not an expert in CT, but I do know enough to know what Landauer's principle is (and that it's been plausibly challenged.) I also know there's some crazy stuff in there, like descriptive complexity theory's link between Existential SOL and NP-Complexity.
Do you have any complexity theory papers that deal with this specifically? I've only ever seen that kind of work done in mathematical logic. Genuine interest in reading the CT approach.
> it usually looks like what you would expect out of a Turing machine's simulation of it
You could assign it any cost model you want. Often this doesn't make a difference (as speedup theorems and other "distracting details" mean that most classes are intended to be separated by exponentials), but it is true that people are typically more interested in cost models that are more relevant to the physical universe (although number of reductions is very much the cost of focus in proof complexity). Indeed, relevant discoveries in physics yield corresponding computational complexity research, as in the case of quantum complexity (https://en.wikipedia.org/wiki/Quantum_complexity_theory).
> Do you have any complexity theory papers that deal with this specifically?
A Google Scholar search for "proof complexity" will show you many papers as well as a number of books.
The point that what gets published adapts to the state of the art is fair. Personally I think its adaptations for ANNs and interaction nets are more interesting than the quantum ones. Once it has to account for the topology of the computer, or deal with non-atomic and continuous substrates, it changes a little bit. But this is orthogonal. The point wasn't that you can't make it adapt, I hope it doesn't come across like I'm dismissing the field and its importance. What I'm trying to say rather specifically is that it's not it's not about studying the nature of computation. It lives at a different level of theory than that. It might just be a disagreement on the article being used. I think it would be fair to say it studies a nature, just not the nature, but that would be true of all fields of CS.
I would say that it very much is about the nature of computation, but computation itself has always been tied to the physical. The physicality (even hypothetical physicality) is exactly what separates theoretical computer science from pure mathematics. Even the two primary resources that complexity measures - time and space - are tied to physical quantities.
The distinction is certainly a fair one to make. The etymological root of computation is "done with mental labor", by way of "to clean a financial ledger" (although weirdly, this isn't the real root. Putare is botancial pruning. Computare is a metaphor created after it spent some time on the semantic treadmill.) Grounding it to the physical is perfectly sane, because that's historically how it's been used.
That being said, a counterargument to press against this is that complexity theory doesn't restrict itself to physical or hypothetical physicality in its totality. As I mentioned, there are swaths of complexity theory work which bound quite far afield. The higher orders of the field are decidedly not-physical at all (and pedantically, hypercomputation isn't strictly computation). Of course even beyond this, we're still not studying the nature of computation, we're studying the cost. While you might say that of course these things are tied thanks to physicality (I wouldn't agree that they're equivalent on this basis, but I don't think that's an interesting semantic argument), I did also mention Landauer's principle being plausibly challenged, which is further problematic for conflating the two. Computation being reversible where entropy isn't doesn't explode complexity theory, but it does drive a wedge between information in a computer-sense and information in a thermodynamics sense. At that point, we don't have the claim in the first place, it's just a false friend. Something to consider.
I'm not saying computational complexity restricts itself to the physically realisable, I said it's tied to it and so there will be more papers on models that have some correspondence with physical reality.
> we're still not studying the nature of computation, we're studying the cost
That is one way of studying the nature of things, sort of like the use of the Hamiltonian in physics, especially if you're interested in problems and classes of problems, and their broad similarities and differences via reductions, rather than in specific computations.
> The higher orders of the field are decidedly not-physical at all (and pedantically, hypercomputation isn't strictly computation).
Hypercomputation isn't really a big part of the main thrust of complexity theory, but (computable) oracles do very much play an important role in complexity theory, even in its lowest complexity classes, such as in the relativisation barrier, which shows that some proof techniques cannot separate P from NP.
> I did also mention Landauer's principle being plausibly challenged,
That's not complexity theory, at least not the standard theory, which treats time and space (or circuit size) more abstractly than concrete physics. There are, however, theoretical reversible models, just note that they don't yield different "classic" deterministic complexity classes (i.e. they do not yield exponential differences).
Again, all fair. To simplify, my points are to demonstrate a confounding nuance, simple counterexamples where the arguments don't hold.
> Hypercomputation isn't really a big part of the main thrust of complexity theory
I don't mean to imply that it was, though the results are actually relatively important elsewhere.
> That's not complexity theory, at least not the standard theory, which treats time and space (or circuit size) more abstractly than concrete physics.
More, but not totally abstractly. Steps and cells being vacuous primitives, they're not literally space-time, but within orthodoxy there's absolutely a partial morphism that's implied. That's why they're named like that. You are supposed to have them live close together in your head.
>There are, however, theoretical reversible models, just note that they don't yield different "classic" complexity classes
I know that some don't, but for example quantum models to use your own example, while not technically reversible in the absolute sense, do possess some reversibility capability and do derive different complexity classes.
I think it's very obvious that there should be reversible computational models which yield different complexity classes from the typical ones. To me for a field to qualify as a study on the nature of computation, it should probably be able to design one totally a posteriori, if in a higher order language. Complexity theory might be invoked in such a construction, but it's not the one doing the building. It's one of many in an orchestra.
Here's a question I have, since you do seem pretty well versed on CT. Universal quantification over complexity classes of first-order systems, used anywhere in the abstract?
> Universal quantification over complexity classes of first-order systems, used anywhere in the abstract?
I'm not sure what you mean by "complexity classes of first-order systems" and by "in the abstract".
But it seems like you're asking about the intersection of computational complexity and formal systems, and there's definitely work there. I already mentioned proof complexity, which analyses the number of deduction steps needed to prove something in various formalisms, and there are famous undergrad-level examples, such as TQBF (https://en.wikipedia.org/wiki/True_quantified_Boolean_formul...). But an intersection that is of more interest to me, as I'm interested in software correctness, is that of the model-checking problem.
Now, many people are confused whenever the model checking problem is discussed, because they confuse it with model checkers, which are a set of algorithms intended to solve the problem, but complexity theory is typically interested in the inherent difficulty of answering problems regardless of the algorithm used to do it. So the model checking problem is that of determining whether a formula in some formalism implies another formula, and its inherent complexity exists regardless of whether this question is answered via a formal proof or by some technique involving the logic's semantics. In the context of software verification, the model checking problem is that of determining - by whatever means - whether a program satisfies some non-trivial property.
Philippe Schnoebelen has some papers on the model checking problem in temporal logic (https://lsv.ens-paris-saclay.fr/Publis/PAPERS/PDF/Sch-aiml02..., https://lsv.ens-paris-saclay.fr/Publis/PAPERS/PDF/DLS-jcss-p...). One of his findings that I've found most interesting with regards to programming is that programming languages cannot, in general, make answering the question of whether a program satisfies some property any easier. This result is surprising. The reason is that without a programming language, we could describe a program as a huge state transition graph (this is called a Kripke structure). In that representation, it's been proven that verification is linear in the number of states, i.e. there is no general approach that is faster than brute-force. Now, the size relationship between a program written in a programming language and its Kripke structure is easily exponential or more, so if there were no algorithm that's better, in the worst case, than a brute force of the Kripke structure, then obviously verification is intractable in the size of the program. However, the number of Kripke structures of size N that have a succinct representation in some programming language is far smaller than the total number of Kripke structures of size N. So it could have been the case that analysing programs would have been easier than analysing their Kripke structure (while ignoring their representation in the language). But Schnoebelen proved that this is not the case.
He also proves that program decomposition (and verification of each component separately) cannot, in general, make verification any easier (i.e. the model checking problem isn't FPT in the number of program components).
These results are far more recent than the hopes expressed in the seventies and eighties that we'll be able to prove the correctness of all/most/many programs we write, and indeed, even though the results talk about the worst case, what we've seen in the last few decades is that the power of program verification indeed behaves more like the worst case than something far from it. The gap between the size of programs we can verify and the average size of programs we write has only widened (what saved the day has been the effectiveness of unsound methods, but that's a whole other discussion).
> you just do all that manual work with environment variables
You really don't anymore. For the past several years, Java's GCs mostly pick the right settings automatically, except for heap size, which will be taken care of soon (https://openjdk.org/jeps/8377305). The reason heap size isn't automatic is that with moving collectors it determines the CPU/RAM tradeoff, and doing that in a more natural way isn't trivial, but we have the algorithm now and will merge it soon.
> or making sure to "pick the right collector for the job"
There are really only four options, most of which are easy to choose among: Parallel for batch jobs where only throughput matters, ZGC for interactive applications where latency matters a lot, and then consider either G1 or Serial if there's a problem with those choices.
As someone who's worked for a long, long time solving manual memory management issues, the amount of effort required isn't just in a different ballpark, but in a different city. Sure, spending a few hours a year to reconsider your settings isn't nothing, but it isn't even remotely in the same category of pain with manual memory management (or even automatic memory management, but with malloc/free underneath).
Concretely, what are current tail latencies, worst case?
Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Most servers I work with run on big machines and are the only process, so figure a 100-250GB heap that lives for months, all async, small requests, so insane amounts of Future and String allocation spam.
Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Also, is there any work on statically enforcing data race freedom in Java? That’s a bigger rust selling point than memory safety for me. I think swift has done some interesting work in that space. It would be nice to get those sorts of safety properties without manually writing borrow checker annotations.
As the OC, I think my view is somewhere in the middle - I am neither as optimistic about it being "great now" nor do I think that "rewrite in C++" 10 years ago was easier.
My reason for disagreeing with the former view is that improvements in physical RAM available and tendency towards smaller workloads have allowed many Java (or other GC runtimes) to essentially "fix their problems because hardware got better". So you can waste more RAM, waste more cycles, but "it doesn't matter", and likely it is fine in many cases - but it's is not the same thing as claiming the GC algorithms are responsivle for that outcome. We have been 3 years away from GC solving memory management for at least 30 years.
My reason for disagreeing with the latter view is that for those who don't have 100-250 GB long-lived heaps (or whatever the contemporary version of that is), the pain level is far lower than rewriting in C++ or Rust, or likely the pain level of hiring enough engineers who can do either. It's a completely different engineering culture.
Just to be clear, the main reason for the use of moving collectors in the first place is to waste less cycles on memory management (otherwise we wouldn't use them). They exist to serve as an optimisation.
> We have been 3 years away from GC solving memory management for at least 30 years.
It's now 3 years in the past (since Generational ZGC); e.g. see https://netflixtechblog.com/bending-pause-times-to-your-will.... Of course, it doesn't solve all imaginable memory management issues, but in practice it makes it a non-issue for a large class of interesting and very common programs.
Maybe I should've mentioned at the start that I've implemented several GCs and worked on several Java VM implementations, so I am generally familiar with the tradeoffs between GC algorithms and other runtime details.
Even in the very positive blog you linked, you see statements like
* "ZGC has a fixed overhead 3% of the heap size, requiring more native memory than G1. .." and
* "Reference processing is also only performed in major collections with ZGC. We paid particular attention to deallocation of direct byte buffers, but we haven’t seen any impact thus far. This difference in reference processing did cause a performance problem with JSON thread dump support, but that’s a unusual situation caused by a framework accidentally creating an unused ExecutorService instance for every request."
This was my point about how this sort of thing is a type of manual memory management.
As for waste more RAM, waste more cycles wasn't a statemnt about whether a particular GC is better-performing for certain situations, but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
You're right that managing any resource that isn't just Java heap memory requires manual management, but it isn't what we normally mean by memory management. The two have been somewhat tied together traditionally through reference processing (in the sense of reference queues), which is generally something we now discourage in Java programs, and may be deprecated and removed altogether at some point. It's traditionally been used as a convenience. The framework used in that post is, indeed, based on an old library that relies on manual or reference-processing-assisted management of non-heap resources.
> but that the overall improvement likely has more to do with improvements in CPU speeds and RAM size, than the latest GC version (which tends to simply make a different set of engineering tradeoffs).
Well, the biggest improvement has been the creation of a new "pauseless" collector, ZGC, with a novel GC algorithm (at least for OpenJDK), which does _zero_ GC work in STW pauses, i.e. no scanning, no marking, no moving. In particular, even roots, including stacks, are processed entirely concurrently with the program. The main practical impact of that has been saying goodbye to GC pauses, and getting low latency, that is perhaps even more predictable than malloc/free (and obviously, still has higher throughputs in a large class of interesting programs). The tradeoff is the usual footprint tradeoff, which is the core of moving algorithms, as well as more CPU cycles compared to STW collectors (but again, still less than malloc/free in many programs). The additional CPU can, of course, be compensated for with an even larger heap.
The general idea is to use RAM chips as hardware program accelerators, but in the past latency was also something you had to sacrifice, and this is no longer the case today.
It definitely was easier for the projects I worked on, but they are exactly the use case where the heap is long lived and most of the machine.
I’ve also worked on systems with lots of small processes, and the operational issues that creates dwarfs GC problems: It takes one middle tier machine, and adds 64-128 network boundaries, and also creates an extremely difficult static memory allocation problem.
I know people do it anyway, but it’s rare that they can articulate a decent technical reason for it, and it wastes something like 90% of the hardware (even in carefully optimized code bases / deployments).
> Concretely, what are current tail latencies, worst case?
Well under 1ms for ZGC (to the point that OS-caused hiccups are of similar magnitudes).
> Ten years ago, “rewrite in C++” was definitely easier than getting the Java GC to stay up under server load.
Both could have been hard in some cases, but open-source "pauseless" GCs are only 3 years old (and all of the JDK's GCs are nothing like what they were ten years ago).
> Optimizing that stuff away in Java is harder than writing Rust, so assume idiomatic Java.
Quite the opposite. Performance issues due to memory management are, in practice, more serious in Rust than they are in modern Java.
> Also, is there any work on statically enforcing data race freedom in Java?
There isn't much demand for that atm. If we see growing demand, we could prioritise it.
In rust, I usually just make sure stuff is not Box<>, and try to reuse buffers. That generally gets the memory allocator completely out of the way (except for async).
The remaining allocator performance problems are mostly due to it zeroing allocated memory unless I use unsafe. Is java able to stackify most new Object calls and elide default initialization of object members these days?
I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS. Could you be seeing selection bias in your user base?
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
Escape analysis in OpenJDK will stack allocate values where it can show it is safe to do so. Project Valhalla is also reducing the memory footprint of objects.
As for thread safety, that is more of a language concern than a runtime one. Amongst JVM languages Scala is leading here AFAIK. Its "capture checking"[1] provides thread safety (e.g. [2]) and actually covers escape analysis as well. On Scala Native (the native code backend for Scala) capture checking can be used for safe stack allocation and safe arena allocation.
> In rust, I usually just make sure stuff is not Box<>, and try to reuse buffers. That generally gets the memory allocator completely out of the way (except for async).
You say "just", but this is easy when programs are small. The problem is that this gets harder and harder and harder as programs grow large (the whole point of the JVM's design was to address the performance issues that plague large C++ programs). E.g. someone who works at one of the world's largest tech companies just told me that they have problems with Rust programs spending 30% of their CPU on memory management even when they're as small as a couple hundreds of thousands of LOC.
> Is java able to stackify most new Object calls and elide default initialization of object members these days?
No, the general idea is to just make memory management efficient (although some objects are "stackified" and the compiler will elide zeroing when non-defaults are passed to a constructor). Now, I say "just", but this used to come at the cost of GC pauses and larger footprint. Now it only comes at the cost of a larger footprint.
But there is a definite choice here when it comes to performance. Low level languages give you control that means performance is attained through manual effort. Java takes away control to improve effort-per-performance. Roughly speaking, these tradeoffs mean that when programs are small and the extra effort is manageable, low-level languages are hard to beat, but when programs are large, it is Java that is hard to beat.
> I’m surprised to hear there is no demand for compiler enforced/facilitated thread safety in Java. That was a major pain point in all the Java code bases I’ve worked with in the past, and is a headline safety feature for rust (which goes even further and enforces aliasing rules) and JS.
This used to be a bigger problem when locks were the main mechanism for sharing data among threads. Now, with the wide selection of concurrent data structures, such problems don't occur as much. I'm not saying they don't occur at all, just not frequently enough to become a major priority.
Also, safe Rust's data-race freedom comes at the cost of requiring unsafe for benign races, which are not uncommon in concurrent algorithms (i.e. it excludes even "good" races). This may be fine in languages whose view on performance is "with enough effort you can get good performance", but, as I said, Java is about making more "naive" programs fast with little effort.
It is fair that there are many ways to be slow in any number of programming languages. I'm surprised to hear "Rust programs spending 30% of their CPU on memory management even when they're as small as a couple hundreds of thousands of LOC", although I can visualize some unique workloads where that's unavoidable irrespective of language & runtime.
You say unavoidable, but moving collectors are designed to reduce CPU at high allocation rates by increasing the heap size. Generational moving collectors have a pathological case - a high allocation rate of long-lived objects - but that's quite hard to get yourself into by accident. Their main downside (besides the inherent increased footprint) used to be unpredictable long pauses, which could have a very high impact on tail latencies, and that's just gone with ZGC.
Generational moving collectors are a very powerful memory management optimisation, but because they necessarily require some "interesting" FFI layer between the ordinary heap and any passing of pointers between the program and the hardware/OS - the very thing low-level languages are designed not to have - this is a powerful, general optimisation (not perfect, but extremely useful in a wide class of programs) that is not available to low-level programming languages. And it's not the only one, BTW. JIT compilers are also designed for "global" average-case optimisations at the cost of precise low-level control over the worst case, which is another thing that low-level languages trade away.
In the most simplistic way, I would say that the precise control that low-level languages are all about helps their performance when programs are small (and can be manually optimised globally) and hurts their performance as programs get large. They have to give up on some optimisations that come at the cost of ceding low-level control, and that includes moving collectors.
When I say "unavoidable" I mean exactly things like your "a high allocation rate of long-lived objects (as one example), where the work that has to be done requires a significant number of CPU and memory cycles spent on memory management (manual or automatic).
When I say tradeoffs", I mean exactly things like "inherent increased footprint", or my earlier "wasting RAM" point.
At this point, I'm not sure that we're disagreeing about these details, but rather what we make of them... I view them as just years of continued tuning of tradeoffs (heap size vs CPU cycles vs memory cycles), while you them as major breakthrough that makes garbage collection much more desirable. Is that fair?
A high allocation rate of long-lived objects is not easy to do. Object "death" rate has to equal the allocation rate, so a high allocation rate of long-lived objects means that somehow you get to allocate, say, 1 GB/s of data that is kept for a long time and is discarded at a rate of 1 GB/s.
> When I say tradeoffs", I mean exactly things like "inherent increased footprint", or my earlier "wasting RAM" point.
Well, that is a real tradeoff, but there's a reason why it's a very attractive one for a huge class of applications. There are two ways of looking at this, which amount to the same thing:
1. Because both RAM and CPU are needed for computation, what matters isn't each of their utilisation values separately but only the more constraining or impactful of the two.
2. Because CPU is needed to use RAM, every CPU cycle you spend effectively takes away some other program's ability to use RAM.
This means that for any amount of CPU utilisation, there is some amount of RAM that is effectively free (i.e. has no additional impact), and the more CPU a program consumes, the more RAM it can consume without it making additional impact. It's easiest to see in the extreme case of a program using 100% CPU: no other program can make progress, and so it doesn't matter how much of the available RAM your program is using - it effectively captures all of it whether it uses it or not. But this scales to any amount of CPU utilisation (not quite linearly). What wastes RAM is not using that "free RAM" to reduce the dominant resource, CPU. And what further determines the RAM/CPU "exchange rate" is the RAM/CPU ratio offered by the hardware, which is more RAM-heavy than some appreciate (it is very hard to find a metal or virtual deployment with less than 1 GB of RAM per core these days - taking into account partial cores in virtual machines - outside of embedded devices).
This means that if you have a memory management algorithm that uses more RAM to help reduce CPU as CPU utilisation rises, that's usually a good thing. And moving collectors work exactly like that. The heap overhead in a generational moving collector is only a function of the allocation rate, and a high allocation rate also means high CPU usage.
My colleague, the main developer of ZGC these days, gave a keynote about this very subject at ISMM: https://youtu.be/mLNFVNXbw7I
> I view them as just years of continued tuning of tradeoffs (heap size vs CPU cycles vs memory cycles), while you them as major breakthrough that makes garbage collection much more desirable. Is that fair?
I say that for many years, the main practical, most "felt" tradeoff of moving collectors has been their STW pauses. With pauses eliminated, there is a qualitative change in the attractiveness of moving collectors, making them more appropriate than other memory management techniques for an even broader class of applications than before. Previously, applications that were very sensitive to tail latencies didn't want moving collectors; now, the tail latency is no longer an issue (unless your application's tail latency tolerance is such that a realtime OS is needed). In other words, I'm saying that moving collectors' most impactful tradeoff is now gone.
> A high allocation rate of long-lived objects is not easy to do.
This is exactly what async programs do on the hot path. Consider a 1M request per second process holding 64K of buffers per request. That’s 64GB of allocations per second. Now, assume the requests hit a remote database with 10ms latency. That’s 640MB of live heap in steady state, which ends up in the “long lived” part of most garbage collectors.
Using RAM to save CPU is exactly the wrong tradeoff when such a system becomes CPU bound.
It’s almost always the case that it is CPU bound due to an incoming request spike or elevated retry rates on the backend. Those tend to pile up, creating a 64GB/sec leak.
The alternative is that the system is CPU bound because the heap is large. This is also very common. Unless each collection takes less work as the heap increases in size, backing off the GC rate to free CPU instantly drives the system into metastable failure, where the GC becomes more expensive because the GC is expensive.
Instead of reasoning about this all the time, it’s much easier (for me, granted, I am not a typical java developer) to just jam the CPU intensive work on a low priority event queue so that it uses 100% CPU but never blocks low latency stuff, or things about to retire requests. (Or, stick it in a dedicated but small thread pool if I can’t touch the async event loops).
This ends up being easier to deal with than java, since everything is thread safe, allocations are predictable, and there are CPU escape hatches I can use.
C++ lets me use smart pointers that have exactly the semantics I want, and that are memory safe but racy in practice. Rust makes them actually memory and thread safe, but sometimes adds useless copies, initializations and thread synchronization (or requires unsafe).
> That’s 640MB of live heap in steady state, which ends up in the “long lived” part of most garbage collectors.
It won't, because that is exactly the thing good moving GCs detect and size the young-gen accordingly.
> Using RAM to save CPU is exactly the wrong tradeoff when such a system becomes CPU bound.
Did you mean to write something else, because it's pretty obvious that it's the right tradeoff? If something is CPU-bound, you want to reduce the CPU usage.
> Unless each collection takes less work as the heap increases in size, backing off the GC rate to free CPU instantly drives the system into metastable failure, where the GC becomes more expensive because the GC is expensive.
The whole point of moving collectors is that each collection takes the same amount of work, but you need to do it less frequently as the heap rises. So yes, as they heap grows, moving GCs are supposed to work less. The heap grows as a function of the allocation rate while the CPU devoted to memory management remains the same. That's precisely the optimisation that moving collectors bring.
> Instead of reasoning about this all the time
The whole point is that the GC is what "reasons" about this for you.
> it’s much easier (for me, granted, I am not a typical java developer) to just jam the CPU intensive work on a low priority event queue so that it uses 100% CPU but never blocks low latency stuff, or things about to retire requests.
That's orthogonal. You can do that at least as easily in Java.
> This ends up being easier to deal with than java, since everything is thread safe, allocations are predictable, and there are CPU escape hatches I can use.
Thread safety is orthogonal, and now with ZGC, memory management in Java is more predictable than malloc/free allocators.
> C++ lets me use smart pointers that have exactly the semantics I want, and that are memory safe but racy in practice. Rust makes them actually memory and thread safe, but sometimes adds useless copies, initializations and thread synchronization (or requires unsafe).
Yes, and it's also less efficient and less predictable in the memory management work as programs grow larger.
Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek.
These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
This article is from last year, but suggests they’re still off by an order of magnitude:
If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No. Every garbage collection in Java relocates objects. Compared to malloc, memory fragmentation in long-lived processes is less of a concern. Freelists track only large segments of available memory. The allocator reserves a segment per thread and simply advances a pointer. Small short-lived objects are never visited by the collector. Instead, live siblings are relocated elsewhere before the entire segment is reclaimed.
The above holds true for all of the collectors. The difference is how they deal with concurrent changes to object pointers by the application. Generally, stopping the world uses less net CPU than the memory barriers required by G1GC and ZGC, but most applications are willing to provide more memory and CPU in exchange for shorter pauses.
> Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph).
> These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses.
2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no.
> This article is from last year, but suggests they’re still off by an order of magnitude
The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently.
Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today.
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
> Go, Java, C#, and Python use garbage collectors. This makes them easier to use but slower and less predictable.
It does not. The term "garbage collectors" covers a whole spectrum of algorithms, some might slow you down (though not for the reason you may think) while others were invented to speed up memory management beyond that of C++, in exchange for other tradeoffs. Python's (mostly) refcounting GC is actually closer to C in its memory management overhead than to either Go or Java. It's also not what makes Python slow. Go uses a mark-and-sweep collector to find a balanace between speed, FFI, and footprint. Java uses moving collectors, which are faster - and some of which are even more predictable - than memory management in C++. That's because Java aims to offer better performance than C++ in large concurrent software, where low-level languages tend to suffer from various overheads due to their requirement for low-level control (Java trades off some performance in smaller programs, but mostly it trades of startup time and footprint). Moving collectors (but not refcoting collectors or mark-and-sweep collectors) are an optimisation over free-list approaches, not a compromise for convenience.
So it is true that slow programming languages tend to use some kind of GC, but that's not what makes them slow, nor does it make the super-fast languages that also use a GC (often of a very different kind) any slower. The range of languages that use GCs covers everything from the super slow to the super fast.
I don't understand how you can claim that using a GC does not make a language slower and less predictable.
Running a GC takes time, pollutes the cache, and is often run at an unpredictable time. Sure, the GC is not necessarily the SLOWEST thing about the language (python), but it's not helping, either.
Yes, but for a moving collector that's less time than it takes to run malloc and free. The interaction of a moving collector with most object is bump allocation when they're allocated (similar to stack allocation) and... that's it. The GC never sees them again, scans them again, or is even aware of their existence (moving collectors don't have a free operation). Overall, moving collectors (but not other kinds of GC) reduce the work of memory management compared to malloc/free.
In low-level languages we try to avoid doing a lot of malloc/free not because heap memory management is slow in general, but because that approach to memory management is slow. Moving collectors are an optimisation designed to make heap memory management fast, but it requires that (nearly) all pointers be movable, something that low-level languages can't do because they have constraints that are more important to them than speed (you can't interact with the OS or hardware directly, i.e. without an FFI API, if your pointers are movable, and such direct interaction is the point of low-level languages).
That moving collectors (NOT the GC Python has; NOT the GC Go has) can, in principle, make heap memory management cheaper than stack allocation has been well known since the eighties. But until recently they had excellent throughput (somewhat similar to arenas) but potentially long pauses. It was only recently that they were made "pauseless".
> and is often run at an unpredictable time
How much work malloc and free need to do is also unpredictable, and a modern pauseless moving collector like ZGC spreads the work needed for memory management more evenly than malloc and free.
> Sure, the GC is not necessarily the SLOWEST thing about the language (python), but it's not helping, either.
There is very little resemblance between CPython's GC and Java. Python's memory management is closer to C's than to Java's. GCs cover such a wide spectrum of algorithms that it doesn't make sense to talk about them as a single category as far as performance tradeoffs are concerned.
I've written two separate moving collectors for dynamic language runtimes, as well as done significant work in realtime 3D graphics, and what you're saying mostly smells like bullshit.
> The interaction of a moving collector with most object is bump allocation when they're allocated (similar to stack allocation) and... that's it. The GC never sees them again, scans them again, or is even aware of their existence
I mean .. sure, but, a copying GC eats somewhere on the order of 10% of your total memory bandwidth just copying shit around. I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
> Moving collectors [...] requires that (nearly) all pointers be movable, something that low-level languages can't do
Completely false. You have to do some manual bookkeeping in C++, Rust, Zig, whatever, but you can do it, and in fact many commercial GCs do (V8 is a good example).
...
The rest of what you said is just empty-sounding claims that I'm not going to address. I looked at the single paper that you linked in another comment, from the 80s, which is hardly relevant on modern hardware.
Please, if you're going to make the claims you're making, back them up with hard evidence. I've looked, and the overwhelming majority of papers out there claim that GCs are slow, memory hungry and, generally, a waste of time.
> I've written two separate moving collectors for dynamic language runtimes, as well as done significant work in realtime 3D graphics
That's nice. I have ~25 years of experience with large C++ software, including hard and soft realtime systems, and I now work on the JVM.
> I mean .. sure, but, a copying GC eats somewhere on the order of 10% of your total memory bandwidth just copying shit around
Good moving collectors are designed to copy very little. That's the entire purpose of generations.
> I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
In theory. When you're in charge of a >2 MLOC C++ system, maintained by a large team for well over a decade, you find that these optimisations are very costly. Plus, you commonly find large C++ software that needs sophisticated malloc/free allocators for acceptable performance (BTW, some of those allocators are almost the same size, in LOC, as that of ZGC, probably the world's most sophisticated moving collector).
That's why, where I used to work and oversee large projects, we migrated pretty much all systems (mostly soft-realtime defence software) to Java from C++ - for better performance. One of the goals of the JVM was to tackle the familiar performance issues that plague large C++ programs.
And BTW, using arenas isn't so easy when programs get large and sprawling, or even in general. Zig definitely makes that much easier, though.
> Completely false. You have to do some manual bookkeeping in C++, Rust, Zig, whatever, but you can do it, and in fact many commercial GCs do (V8 is a good example).
What I said is completely true, but you may have misunderstood it. You can, of course, combine moving collectors with things that expect stable ones in the same process. In fact, you absolutely must, because at some point in the stack you need to talk to the OS and/or hardware, and they expect stable pointers. But you have to have some distinct FFI layer between the two, and the entire point of low-level languages is to be at the lower level.
> Please, if you're going to make the claims you're making, back them up with hard evidence. I've looked, and the overwhelming majority of papers out there claim that GCs are slow, memory hungry and, generally, a waste of time.
I don't know what you've read, but that is very clearly not the consensus among memory management experts. My "claims" are pretty common industry knowledge, and why the majority of performance-critical large software has migrated away from low-level languages over the past couple of decades, and the trend continues. I'm not trying to change anything, I'm just explaining why the industry is doing what it's doing to those who may not be familiar with large, long-lived software.
It's funny, but 25 years ago, the people who doubted the amazing performance-per-effort of moving collectors and JITs were those who (like me) had not used those technologies and were mostly familiar with low-level languages. These days, it's the people who have little experience developing and evolving large and complex software in low-level languages (TBF, there's much less such software written in low-level languages these days) that believe the low-level languages are inherently fast.
Having said all that, when programs are relatively small and/or not very concurrent, the effort required to match or beat Java's performance in a low-level language through careful manual optimisation is sometimes worth it. In large software, it gets harder and harder.
> I can guarantee that if you use sane allocation strategies (arenas & freelists, pools, whatever) you spend <1% of your total system resources fucking around with memory allocation.
I just spoke with someone on the performance team at one of the world's largest tech companies who told me that some of their larger Rust programs spend 30% of their CPU on malloc/free. Of course, it's possible in principle to reduce this given enough effort, but this is identical to the experience we've had with C++ for decades: When programs are small, it's easy to get good performance, but as they get larger, the areas where low-level languages have intrinsic inefficiencies (such as dynamic memory management) tend to become more pronounced in practice and the programs are not so easy to optimise.
The quoted text says "slower". It does not claim that GC makes those languages slow overall.
Are you arguing that GC is not inherently slower than other memory management strategies (e.g. the Rust approach)? Or just that the cost is not worth optimizing away?
Of course GC is not inherently slower than other memory management strategies. Not only because GC is not a "memory management strategy" but a wide spectrum of them, but also because some GCs are used to speed up memory management compared to low-level languages. Dynamic heap allocations in low-level languages is often minimised because it is slow; some GCs are used to solve this problem.
Have the Rust and Zig communities been outright lying to me? I know very little about the details. I genuinely thought that the overhead of GC was a well-established tradeoff in language design.
That some specific kinds of GCs, specifically moving GCs (of the kind used in Java and .NET, but not the kinds used in Python or Go, which are also very different from each other) can be highly efficient, at least in principle, has been well known since the eighties (see Garbage Collection Can Be Faster Than Stack Allocation, 1986 [1]). However:
1. Moving collectors have long been more efficient than malloc/free but only in the total work devoted to heap management, i.e. throughput. Moving collectors that also offer predictable low latency are much more recent. In fact, the first open-source, production-quality, high-throughput, low-latency moving collector is less than three years old [2].
2. Much more importantly - and this is something that we experienced low-level programmers have known forever but people without much experience in low-level programming seem to not know these days is that low-level languages are not optimised for maximum performance. They're designed for maximum low-level control. When programs are small, when they're not heavily concurrent, or when the hot path is relatively simple, this low-level control can, indeed, translate to very high performance. But when programs grow larger or more concurrent, low-level control can actually make some optimisations harder. In particular, moving pointers, which are required to enjoy the optimisation offered by moving collectors, is not compatible with the low-level control needed in low-level languages, and these languages prioritise low-level control over everything else, including performance (as low-level control is their primary purpose). And this is not the only example of optimisations that these languages make harder. This is why large and/or concurrent programs have largely migrated from C++ to Java and C# over the past few decades, and this trend isn't reversing. Again, for smaller and/or less concurrent programs, low-level languages still offer excellent performance in expert hands, provided you invest sufficient effort into manual optimisation.
3. The optimisation offered by moving collectors isn't free. Until recently, you had to pay in unpredictability and high latency (which is still the case in all languages except Java), and since the algorithm uses RAM to reduce the CPU needed for heap management, it does necessarily require higher footprint. You can enjoy a similar optimisation that turns RAM into free CPU in Zig by using arenas (this is harder to do in C++ and Rust). Zig's arenas are even more efficient than moving collectors, but they do require more effort, and they're less general.
[1]: https://www.cs.princeton.edu/techreports/1986/045.pdf Note that in practice, moving GCs are not quite as efficient as stack allocation (let alone more efficient), but heap allocations in Java are not as expensive as heap allocations in C/C++/Rust/Zig that utilise malloc/free. This is why in these languages we try to avoid heap allocations on the fast path.
The JVM goes for performance, too, only with an emphasis on the performance of larger programs. It's designed to address some of the serious performance issues that large C++ programs tend to suffer from (I originally made the switch to Java because it was getting hard to keep the large C++ applications we were working on fast enough; Java does some optimisations that are hard for a C++ compiler).
One thing we C++ programmers know, though, is that direct control over instructions helps performance in small programs but can hurt it in larger programs. E.g. it is really hard to enjoy the performance benefits that moving collectors and JITs can bring in large C++ applications.
It's not so easy. The pattern you'd want to try and match or beat moving collectors is arenas, and they're really not trivial to use in C++ (or Rust). The only low-level language that supports them well is Zig. But even then, evolution of the program can often require large architectural changes.
So yes, in principle it's possible to match and perhaps somewhat exceed Java's performance in C++, even in large programs (just as, in principle, it's possible to match and exceed C++'s performance in Assembly), but we don't care about what's possible in principle; we care about what we can achieve with the budget we have. Or put another way, C++ has better performance/effort than Assembly, and Java has better performance/effort than C++ in large programs (perhaps not in all domains, but in important ones). The JVM was designed to (among other things) specifically make it easier to overcome some known performance problems that large C++ programs experience.
> we care about what we can achieve with the budget we have
100% agree on this. This is the last driver for everything else in professional environments.
Related: I think C++ safety being driven by profiles and not the Safe C++ stuff that was proposed violates the economic assumption in so many ways that it was the wrong choice for C++.
I think you've misunderstood, because not only would this have been the right design if it had been in Java since day one as it's in line with the philosophy of the platform, it's also simpler than in other languages.
The idea is that instead of controlling memory layout and referencing directly, you communicate your intent: do you care about this value's identity or not? Do you need atomicity or not? Do you need nullability or not? Once the intent is clear, the compiler is free to choose the most appropriate and efficient layout for the particular value at the particular use site. In other words, you say what semantics you're interested in rather than how to implement them at the lowest level.
This opens up optimisation opportunities that are lost when the programmer directly controls the representation rather than the intent. E.g. in other languages you may say whether you want to pass some value by reference or by value. Here the compiler is free to say, well, if identity and nullability are not needed here, I can either pass by value or by reference, and I'll do whichever is more efficient.
And by the way, your point about early misdesign (whether it applies here or not) also inverts the desired state. Every language makes decisions that will be suboptimal in the environment some time later, and Java certainly has its share (its mutability and nullability default; how it treats serialisation). Rust, for example, was first designed twenty years ago, and some of its fundamental decisions reflect the state of the world at that time (it went all-in on some C++ premises that seemed fine 20 years ago). But since important codebases often outlast the outdatendness of early language decisions (your OS and your browser are running some >30yo code and/or affected by >30yo design decisions), one of the things most important in a language isn't the decisions it makes early on - some will prove "wrong" while your codebase is still alive and kicking - but how well it adapts and evolves. So when you pick a language for an important project today, the language's current state will end up mattering less than how the language evolves in the future. The question asked isn't "will I like this decision today?" but "will I regret this decision ten years from now?" Java is one of the languages with the lowest "regret factor", possibly lowest of them all.
1. Upgrade your JDK for the best performance (as the article says, the slowdown is gone in JDK 26).
2. Don't try to help the GC by pooling objects. Mutating old objects can be expensive, while allocating new ones is cheap (at least for objects that don't do some exceptionally expensive initialisation).
reply