Zen 5 really is nice. I swapped my RAM from a Xeon W Sapphire Rapids machine into a Threadripper Pro 9000 series machine and I get almost double the memory read performance, plus it's a heck of a lot faster in single and multi core performance, and it runs cooler and quieter. Huge win all around. Aside from the price... (I went from Xeon W5-3435X to TR Pro 9985WX, eep.)
I went the opposite way and went back to Intel after many many years on AMD. I snagged a Xeon 654 Granite Rapids ($850) for a new workstation geared towards local inference. I don’t need a ton of CPU cores and all Granite Rapids have 8 memory channels where as you need very high end Threadrippers (9975wx for $4000) for an equivalent due to their CCD design.
As a bonus Granite Rapids is super power efficient and runs much cooler than my previous TR and Ryzens. Intel seems to be doing good things again! My only minor complaint is that P2P doesn’t work on my multiple GPU setup because every PCIe5x16 lane has its own dedicated root to the CPU, but all that bandwidth is useful for MoE models that are offloaded to RAM.
You could get a PCIe switch backplane, stick your GPUs on there and then have them communicate locally through the switch using p2p. For some GPUs you may have to use a hacked driver (google p2p 3090 for instance).
C-Payne makes excellent stuff but it is very pricey and supply is more than a little spotty, I've used pretty much all of his stuff by now and I am a very satisfied customer if I can order what I need, which more often than not is not the case.
A good alternative is ADT, they make excellent boards, there is a 4 and a 5 slot PCIe expander with a 88096 on it that works extremely well. I have two of these connected to my rig with their own power supply and four GPUs in each (and another two in the main machine). It's not exactly a portable affair (to put it mildly). Note that these won't work in a standard PC case due to the slot spacing so you'll have to rig something for that yourself (I use 2020 + some custom 3D printed fixtures).
Yeah, I had to go high-end to get one with enough CCDs to make use of the memory channels. I'm curious to know what all-read bandwidth you see when you run intel mlc on your system. My Xeon W5 was doing ~180GB/s after a lot of tuning, which was very disappointing. The Threadripper does ~320GB/s. From what I've heard, Granite Rapids is supposed to have solved the bandwidth problems that ailed Sapphire Rapids, so your numbers ought to be similar.
I have the complete opposite experience. Got a 9550x with 64gb ddr5 and a fairly high end mobo about two year ago. Just running the memory at stock speed. About 50% of the time I’d reboot and one or both of the sticks would only be detected as 2gb. Would need to do a hard shutdown to get it back.
I eventually gave up and turned off memory context restore and now I just deal with the minute plus (!) time to Post.
Not sure if amd memory controllers are just garbo or what but I’m going back to intel next chance I get.
On the desktop Intel has better memory controllers last time I looked, but it’s not that big a difference. Something in your setup is just broken. Failures will happen with any brand, so I wouldn’t chalk it up to AMD vs Intel… figure out where your problem is and get the faulty part replaced if you can.
I feel you though, regardless of the cause, that is an extremely frustrating place to be.
Yea I’m just bummed because it’s a real expensive time to be swapping memory parts around. Like I said, things seem to just work without memory context restore so I’m fine with just paying the cost when I reboot once a week or whatever. I feel like the fact that disabling MCR fixes it should narrow down the issue to some part, I’m just not sure which.
Did you set it to do memory training on every boot? That's a big stability help.
Should also run Memtest86+ a few times. With some stick swapping you can determine if you have a bad stick or slot.
Yea I’ve done endless googling around the issue and it seems to be a not uncommon issue with ddr5 but there’s no way I’m buying new parts now so I’ve come to peace with it.
I'll probably manage with Mac OS well enough, but my linux distro comes out of the box with all the latest OSS tooling I'm familiar with, plus a package manager, and it has linux cgroups and namespaces that power the container technologies we all know and love.
If I switch to Mac OS, I have to sort out a package manager and install all the stuff that's missing, and when it comes to containers... they're just linux VMs. I'd happily cut out the weird proprietary middleman if I could.
Linux for argument’s sake, may have a few things that are better than Mac OS but Apple being the last vertical computer company from the 1980s, I don’t think they have any interest in using Linux, not after Next, Motorola, IBM, Intel and Nvidia in the past. They don’t need to they appear to navigate thru tech very well in comparison to Microsoft or Intel, for example.
I think people responded that way because you strongly implied he was an unserious vibecoder who was just fooling around, and you called him suspicious as fuck.
In my own professional life, I've found this to be a very divisive statement. For some, it is a sign of wasting time and effort. For others, they use this to describe themselves when they want to do exploration for the goal of finding improvements, without any clear goal because they have a few ideas but none worth putting forward. I've been told to spend time learning AI and have found that saying "Yeah, I'm playing around with it." was the wrong thing to say because it was seen as not doing anything worthwhile. It doesn't matter that I would also say the majority of my tech skills were developed when I was "playing around".
I wonder if this is purely a linguistics breakdown, or if this is tied to some deeper difference in a person's relationship to tech?
Agree it's divisive, and I would argue it speaks more to a person's perception of work vs play more than a relationship to tech. If (the general) you think that play is for children and work is serious biz, then yeah I could see how you wouldn't take someone seriously when they say they're "playing around with it". It's usually not obvious which attitude a person has though without getting to know them a little bit.
Yes, I think it can be a sign of a very deep difference. When they say "spend time learning AI," they mean work through some teaching materials to learn how to replicate what others are doing. This often doesn't result in a deep understanding, but it can be enough to allow them to do their job.
Ironically, people with this mindset will sometimes ask people who they recognize as having strong skills to share their magic secret, which is assumed to be some books they read, videos they watched, courses they attended, etc. If you tell them that experimenting, playing around, etc. is a key element, they may assume you're just selfishly hoarding your fount of knowledge.
I don’t see anything from your parent commenter on the other thread that deserves that classification. On the contrary, while they initially had suspicious of vibe coding, on later comments they are cordial and even admit their own misunderstanding.
What am I missing? Where does “called him suspicious as fuck” come from?
Not to mention the author themselves say "Yes, there's a lot of vibe-coding in many places [...] We'll prune AI slop over time.", and then they both moved on to discussing the actual questions.
The whole "Wow, looks AI" > "Yeah, some of it is, we'll fix it later" was such a small part of the conversation, but then there are countless of other people chiming in about specifically the "Is It Slop Or Not?", rather than the meat of the conversation. And here we are adding even more meta-comments about it.
The thing they don't show is the one we really need, especially because model providers can skimp on quality (run lower quantization, lower kv cache precision, etc) to improve their pricing and performance. I agree that it's probably too expensive to keep running the benchmark, but we need some way to hold the providers to a certain standard, otherwise every user has to discover the problems on their own.
As a user/reader/viewer I absolutely hate Anubis and usually turn around when I see it pop up (at least on my phone where it takes ages to compute), but with stats like that, I get why a site operator would resort to using it.
I think the kernel.org post proves the parent point rather than contradicts it.
> At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html.
14 CPU cores total for running a website like kernel.org is laughable. This is not worth burning cycles in Anubis on client's devices, this is not worth the time of the engineer who worked on it. Provisioning more hardware would have been literally better for everyone.
> But no, let's in fact choose the stupidest possible way of doing it — by rendering everything as HTML commit by commit and then parsing it.
This drives me crazy with so-called SOTA LLMs that have "achieved AGI".
Fable, Sol, Astra, will start by trying to reverse engineer a binary to figure out how something works when software is open source and one search query away.
You let them know it's open source, and they will start using github API instead of just cloning and grepping.
reply