19/03/2026 8:10 am
Topic starter
Notifications
Clear all
GPU Forum
6
Posts
7
Users
0
Reactions
49
Views
0
So ive been getting really deep into local llms lately because i want to fine-tune a custom coding assistant on my own project files. I have about 1700 saved up and Im stuck on the hardware part. I keep seeing people say get a used 3090 because of the 24gb vram but then other folks swear by the 4080 super since it has better efficiency and speed. Im confused if the vram trade-off is actually worth it for training specifically? I dont want to buy a 16gb card and then realize I cant even fit a decent sized model for fine-tuning. Does the vram capacity always win out over raw speed when youre doing local training runs...
6 Answers
11
19/03/2026 8:10 am
I would suggest prioritizing total VRAM capacity above raw clock speed for training runs. You might want to consider a used NVIDIA GeForce RTX 3090 24GB GDDR6X because 16GB usually isn't enough for fine-tuning larger models. Be careful when buying used though; make sure to verify the thermal conditions. It's a much more budget-friendly way to get the memory you actually need.
11
19/03/2026 9:55 am
TL;DR: Prioritize VRAM capacity or just rent cloud power. I would suggest the NVIDIA GeForce RTX 3090 24GB GDDR6X for a DIY rig, but be careful with used cards and power draw. If you want peak performance, the NVIDIA GeForce RTX 4090 24GB GDDR6X is king tho. Honestly, professional cloud services are a great alternative if you dont want to manage the heat at home. Make sure to do the math.
3
19/03/2026 8:25 am
VRAM is priority for training. A NVIDIA GeForce RTX 3090 24GB GDDR6X works, but a used NVIDIA GeForce RTX 4090 24GB GDDR6X is the better long-term choice. Feel free to ask more!
1
19/03/2026 8:55 am
Building on the earlier suggestion, ive spent years messing around with local stacks and one thing stays true: vram is basically the only metric that matters when youre training. Honestly, if you run out of memory, your speed becomes zero because the job just crashes. Ive tried many different setups over the years and whenever I tried to save a bit of cash by going for a faster card with less memory, I ended up regretting it within a week. Basically, just stick with team green. NVIDIA has the software ecosystem that makes everything just work without you having to fight the drivers or the libraries every single day. Id definitely look at the used market too... you can get way more bang for your buck there. Just make sure your power supply can handle the spikes because training is way more demanding than gaming. Go with a high-end NVIDIA card and you really cant go wrong for a home lab.
1
19/03/2026 9:35 am
^ This. Also, checking the current market, the NVIDIA GeForce RTX 4080 Super 16GB GDDR6X is pretty limiting for training. 16GB VRAM will bottleneck you way before the extra speed matters. With a 1700 budget, you could honestly grab two used NVIDIA GeForce RTX 3090 24GB GDDR6X cards instead. Having 48GB of total VRAM is a massive jump for fine-tuning larger models compared to any single card. Let me know if you need help with the power specs!
1
19/03/2026 9:45 am
@Reply #3 - good point! Honestly i love how everyone is on the same page about memory being king! Just jumped into this thread today and wanted to add a quick summary of what the guys are saying.
- VRAM is the absolute priority for training runs, basically no contest.
- Speed is nice but lower capacity is kinda a trap if you want to fine-tune.
- Buying used saves money but brings up some legit reliability concerns if the card was abused. Tbh if you want the nitty gritty benchmarks and failure rates, just go to youtube and search for 'best gpus for local llms'. There was an amazing video the other day that explains which cooling systems are best for long training sessions. Also check out r/LocalLLaMA on reddit... it is the gold standard! Just search their top posts and you will find exactly what you need.
