What are the top workstation GPUs for high-end AI workloads?

Honestly, I feel u on the price of the Ada stuff... it's literally insane. I've been doing AI dev for years and unfortunately, the high-end workstation cards often feel like a rip-off unless you absolutely need ECC memory. For your situation, I'd suggest looking at the NVIDIA RTX A6000 48GB (the previous Ampere gen). It's got the same 48GB VRAM as the newer ones but you can find it way cheaper now. I had issues with the NVIDIA GeForce RTX 4090 24GB cuz of the 24GB limit—once you hit that wall with LLMs, it just crashes and ur day is ruined. The A6000 is sooo much more stable for long training runs. Basically, if memory capacity is the goal, the extra VRAM on the older pro cards beats the raw speed of consumer cards any day tbh. Just my two cents... anyway, good luck with the build!!

For your situation, I'd honestly skip the NVIDIA RTX 6000 Ada Generation if the price is making you sweat. In my experience over the years, running dual NVIDIA GeForce RTX 4090 24GB cards is basically the play if you wanna save cash.

* Dual NVIDIA GeForce RTX 4090 24GB: ~$3,400 total (48GB VRAM pool via NVLink-ish hacks or just splitting loads)
* Refurbished NVIDIA RTX A6000 48GB: ~$3,200 (older tech but stable)

Seriously, the 4090 is literally faster for most SD renders, though it lacks that ECC memory. Still, for a home studio, it's the best value-to-performance ratio imo.

Curious about one thing: what's your power supply looking like right now? Before I suggest a DIY route like the NVIDIA GeForce RTX 4090 24GB or older stuff like NVIDIA RTX A6000 48GB, I gotta know if your rig can actually handle the massive draw from two cards without melting something. honestly, if you're worried about stability, a pro-grade card is usually safer, but it literally costs like three times as much as a consumer setup lol.

sooo i went through this exact same headache last year when i was trying to scale up my home setup for fine-tuning. honestly, after several years of doing this, i've had some reallyyy frustrating experiences with the high-end stuff. i once saved up for months to buy a super expensive pro card thinking it would be the holy grail for my massive datasets, but unfortunately, it felt like a bit of a letdown. the stability was okay, i guess, but the raw speed just didnt justify the massive price tag for my specific workflow. it was kind of a hard lesson that more expensive doesnt always mean 'better' for every ai task. i mean, i definitely learned to look at the raw memory bandwidth more than the marketing fluff.

but i'm curious, what's your actual target for VRAM right now? like, are you trying to fit 70B models or something even bigger? also, are you worried about the power draw if you stack multiple cards? id love to hear more about your setup goals! peace

• Similar situation here - I tried pushing a consumer-grade build too hard and literally almost started a fire... it was sooo scary.
• Stability matters because losing a three-day fine-tuning run is the worst feeling ever and i learned the hard way that your cooling setup is everything.
• Honestly, just get any of the workstation series cards from NVIDIA. They handle the heat way better and wont crash your system when things get intense lol.

Would love to know this too