Which NVIDIA GPU is best for training deep learning models?

In my experience, deep learning basically needs TONS of VRAM to store weights... without it, you'll constantly hit memory errors! I'm still kinda new but here's what I recommend:

• NVIDIA GeForce RTX 3090 24GB - I found one used and it's AMAZING for LLMs!
• NVIDIA GeForce RTX 4070 Ti Super 16GB - GREAT bang-for-your-buck for brand-new cards.

VRAM is highkey most important tbh. gl!

> NVIDIA GeForce RTX 3090 24GB - I found one used and it's AMAZING for LLMs!

This^ Also wanted to add that over the years, I've seen VRAM capacity win every time. Ngl, a used NVIDIA GeForce RTX 3090 24GB is the ultimate bang-for-your-buck move right now cuz of those 24 gigs. If you're gonna buy new, the NVIDIA GeForce RTX 4070 Ti Super 16GB is decent, but you'll hit a wall with LLMs way sooner than you'd think. Basically, VRAM is king... don't settle for 16GB if you can help it!

Works great for me

I totally agree with everyone saying VRAM is the priority, but if you want to maximize your actual training speed, dont overlook the NVIDIA GeForce RTX 4080 Super 16GB. While it has less memory than the 3090, the newer architecture is way faster for computation, especially with FP8 support in PyTorch which can really speed up those CNN epochs you mentioned. Since you are worried about power and heat, definitely dont skimp on the power delivery. Instead of just a basic gold unit, I'd look at the Seasonic Prime TX-1000 1000W 80 Plus Titanium because the higher efficiency really helps when you are pulling 400W+ during 48-hour training marathons. Also, to keep those temps from throttling your card mid-run, a high-airflow case like the Fractal Design Torrent is a literal lifesaver. It basically forces air through the GPU area so you can maintain boost clocks without the fans screaming. Ngl, those long runs are way less stressful when you know your hardware isnt cooking itself!

> I’m also slightly concerned about power draw and whether I'll need a specialized cooling setup

Quick question—what PSU wattage and case are you using? Before I compare the NVIDIA GeForce RTX 4090 24GB vs NVIDIA GeForce RTX 3090 24GB vs NVIDIA GeForce RTX 4070 Ti Super 16GB, I gotta know if your setup is safe for sustained 450W+ loads. I've seen cards literally thermal throttle and crash mid-training cuz of bad airflow... reliability is highkey the most important thing for long epochs!!

Ok so I'm just catching up on this thread and honestly, this^!! Seconding the recommendation above because VRAM is literally the only thing that matters when you're deep into a training run and don't want it to crash. I've tried many setups over the years, and I'm a bit of a cautious builder, so I'd say be super careful with the power draw and heat.

Comparing your options:
- NVIDIA GeForce RTX 3090 24GB: Still the absolute value king for DIYers, but it runs HOT and is power-hungry.
- NVIDIA GeForce RTX 4090 24GB: The dream card, but maybe overkill for a first workstation unless you have the cash.
- NVIDIA GeForce RTX 4070 Ti Super 16GB: Way more efficient, but 16GB VRAM is basically the bare minimum for LLMs now.

In my experience, a used 3090 paired with a solid Corsair RM1000x 1000 Watt 80 Plus Gold Power Supply is the safest "bang for buck" move. Just make sure your case has decent fans... those 3090s get toastyyy. gl with the build!

Tbh I have this exact same issue and it is so frustrating because I have been dealing with this for about four months now and still havent found a clear answer that actually addresses the trade-offs between memory density and power efficiency. I have to politely disagree with the consensus that it is an easy pick, as I am also stuck in this research loop and just cant find a solution that works for my specific local training environment. It is basically a total headache trying to find a reliable path forward...

Great info, saved!