AI Hardware
26 GPUs & accelerators · self-host price guide · cloud rates
21 hardware items · Consumer GPUs
- RTX 3060 (12GB)Budget king
NVIDIA · 2021 · 13 TFLOPS · 360 GB/s
12 GBGDDR6▾$220 usedFits: 7-8B models at 4-bit (Llama 3.1 8B, Gemma 2 9B)
Price (new): $280Price (used): $220FP16 compute: 13 TFLOPSMemory BW: 360 GB/sPower: 170W TDPCheapest entry to CUDA AI. Slow but works. 12GB is the key feature.
- RTX 3090 (24GB, used)Best value
NVIDIA · 2020 · 36 TFLOPS · 936 GB/s
24 GBGDDR6X▾$700 used$0.22/hr liveFits: 14B FP16 / 30-34B 4-bit / 70B 2-bit
Price (used): $700Cloud on-demand: $0.22/hr · RunPod (live)Cloud spot: $0.11/hr · RunPod (live)FP16 compute: 36 TFLOPSMemory BW: 936 GB/sPower: 350W TDPThe undisputed used-market champion. 24GB VRAM at $700 beats almost everything new under $2000.
- RTX 4070 Ti Super (16GB)
NVIDIA · 2024 · 45 TFLOPS · 672 GB/s
16 GBGDDR6X▾$800Fits: 13-14B FP16 / 30B 4-bit
Price (new): $800FP16 compute: 45 TFLOPSMemory BW: 672 GB/sPower: 285W TDPGood perf/W. 16GB is limiting for bigger models — 3090 used is better value.
- AMD Radeon RX 7900 XTX (24GB)
AMD · 2022 · 61 TFLOPS · 960 GB/s
24 GBGDDR6▾$900Fits: 14B FP16 / 30-34B 4-bit
Price (new): $900FP16 compute: 61 TFLOPSMemory BW: 960 GB/sPower: 355W TDP24GB VRAM cheaper than NVIDIA. ROCm support works with llama.cpp + vLLM, but ecosystem is smaller.
- RTX 4090 (24GB)Sweet spot
NVIDIA · 2022 · 82 TFLOPS · 1008 GB/s
24 GBGDDR6X▾$1,400 used$0.34/hr liveFits: 14B FP16 / 30-34B 4-bit / 70B 2-bit
Price (new): $1,800Price (used): $1,400Cloud on-demand: $0.34/hr · RunPod (live)Cloud spot: $0.20/hr · RunPod (live)FP16 compute: 82 TFLOPSMemory BW: 1008 GB/sPower: 450W TDPBest single-card consumer GPU. 2x faster than 3090 at same VRAM.
- RTX 5090 (32GB)Flagship
NVIDIA · 2025 · 104 TFLOPS · 1792 GB/s
32 GBGDDR7▾$2,000$0.69/hr liveFits: 20B FP16 / 70B 4-bit
Price (new): $2,000Cloud on-demand: $0.69/hr · RunPod (live)FP16 compute: 104 TFLOPSMemory BW: 1792 GB/sPower: 575W TDPCurrent flagship consumer card. 32GB unlocks Llama 3 70B at 4-bit on a single card.
- RTX 3070
NVIDIA · 2026
8 GBVRAM▾rent onlyFits: 8GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 3080
NVIDIA · 2026
10 GBVRAM▾rent onlyFits: 10GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 3080 Ti
NVIDIA · 2026
12 GBVRAM▾rent onlyFits: 12GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 3090 Ti
NVIDIA · 2026
24 GBVRAM▾rent onlyFits: 24GB VRAM — 13B-70B quantized
Auto-discovered via RunPod live pricing.
- RTX 4080
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 4080 SUPER
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 5080
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 2000 Ada
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 4000 Ada
NVIDIA · 2026
20 GBVRAM▾rent onlyFits: 20GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX 4000 Ada SFF
NVIDIA · 2026
20 GBVRAM▾rent onlyFits: 20GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX A2000
NVIDIA · 2026
6 GBVRAM▾rent onlyFits: 6GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX A4500
NVIDIA · 2026
20 GBVRAM▾rent onlyFits: 20GB VRAM — small models
Auto-discovered via RunPod live pricing.
- RTX A5000
NVIDIA · 2026
24 GBVRAM▾rent onlyFits: 24GB VRAM — 13B-70B quantized
Auto-discovered via RunPod live pricing.
- Tesla V100
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.
- V100 SXM2
NVIDIA · 2026
16 GBVRAM▾rent onlyFits: 16GB VRAM — small models
Auto-discovered via RunPod live pricing.