Which GPU is best for training deep learning models at home?

Hey everyone! I've been dabbling in deep learning for about a year now, mostly using Google Colab and Kaggle for my projects. While they've been great for getting started, I'm hitting those usage limits way too often lately, especially when I'm trying to fine-tune some smaller LLMs or work with larger image datasets. I think it's finally time to bite the bullet and build a dedicated workstation at home so I can iterate faster without worrying about hourly credits.

The problem is, looking at the current GPU market feels like staring at a puzzle. I’m trying to figure out which card offers the best 'bang for your buck' specifically for training. I know the RTX 4090 is the current king of consumer cards because of that massive 24GB of VRAM, but the price tag is honestly a bit daunting for my current budget. I’ve also seen a lot of discussions suggesting a used RTX 3090 since it also packs 24GB of VRAM and is significantly cheaper on the second-hand market. Is the architectural jump in the 40-series actually worth the extra $800+ for training purposes?

I’m mainly focusing on NLP and some computer vision tasks. My biggest concern is VRAM—I've heard that 16GB (like what you get on the 4080 Super) might become a bottleneck pretty quickly if I want to experiment with larger batch sizes or more complex architectures. Also, since this is a home setup, I’m a bit worried about the power draw and cooling requirements. I really don't want my office to turn into a literal sauna every time I run an overnight training job!

I’ve spent hours looking at CUDA core counts and memory bandwidth, but real-world experience always beats a spec sheet. For those of you who have a local rig, what would you recommend for someone with a budget of around $1,500 for the GPU? Should I prioritize the newest architecture for longevity, or is maximizing VRAM with an older or used card the smarter move for a home learner?