AI Hardware

26 GPUs & accelerators · self-host price guide · cloud rates

Live

Tier

All Edge / On-device Consumer GPUs Workstation Datacenter

Sort

Cheapest first Most VRAM Fastest Newest

23 hardware items · Datacenter

Google TPU v5e
Google · 2023 · 819 GB/s
16 GB
VRAM
$0
$1.20/hr/hr cloud
▾
Fits: Up to 70B in pods (multi-chip)
Price (new): $0
Cloud rental: $1.20/hr (GCP) · only available as service
Memory BW: 819 GB/s
GCP-only. Cheap, fast for JAX/TF workloads. Works with vLLM via JetStream.
NVIDIA T4 (16GB)
NVIDIA · 2018 · 65 TFLOPS · 320 GB/s
16 GB
GDDR6
$900 used
$0.35/hr/hr cloud
▾
Fits: 7-13B 4-bit
Price (used): $900
Cloud rental: $0.35/hr (AWS g4dn)
FP16 compute: 65 TFLOPS
Memory BW: 320 GB/s
Power: 70W TDP
Very cheap to rent. Too slow for production serving but fine for batch inference.
NVIDIA L4 (24GB)
NVIDIA · 2023 · 121 TFLOPS · 300 GB/s
24 GB
GDDR6
$2,500
$0.44/hr live
▾
Fits: 13-14B FP16 / 30B 4-bit
Price (new): $2,500
Cloud on-demand: $0.44/hr · RunPod (live)
FP16 compute: 121 TFLOPS
Memory BW: 300 GB/s
Power: 72W TDP
Modern replacement for T4. Single-slot, low-power — great for density.
NVIDIA A100 40GBSweet spot
NVIDIA · 2020 · 312 TFLOPS · 1555 GB/s
40 GB
HBM2e
$8,000 used
$1.10/hr/hr cloud
▾
Fits: 30B FP16 / 70B 4-bit
Price (used): $8,000
Cloud rental: $1.10/hr (RunPod) / $3.06/hr (AWS)
FP16 compute: 312 TFLOPS
Memory BW: 1555 GB/s
Power: 400W TDP
The de-facto standard for serious AI training + inference. Huge ecosystem. Used price dropping fast.
NVIDIA A100 80GB
NVIDIA · 2021 · 312 TFLOPS · 2039 GB/s
80 GB
HBM2e
$12,000 used
$1.60/hr/hr cloud
▾
Fits: 70B FP16 / 200B+ 4-bit
Price (used): $12,000
Cloud rental: $1.60/hr (RunPod) / $4.10/hr (AWS p4de)
FP16 compute: 312 TFLOPS
Memory BW: 2039 GB/s
Power: 400W TDP
Most rented ML GPU. 80GB fits Llama 3 70B at FP16 on a single card.
AMD MI300X (192GB)Best value
AMD · 2023 · 1307 TFLOPS · 5300 GB/s
192 GB
HBM3
$15,000
$0.50/hr live
▾
Fits: 405B FP16 / frontier models at 4-bit
Price (new): $15,000
Cloud on-demand: $0.50/hr · RunPod (live)
FP16 compute: 1307 TFLOPS
Memory BW: 5300 GB/s
Power: 750W TDP
More VRAM than H100 at half the price. ROCm is decent now — works with vLLM, PyTorch, llama.cpp.
NVIDIA H100 (80GB)Flagship
NVIDIA · 2022 · 989 TFLOPS · 3350 GB/s
80 GB
HBM3
$30,000
$2.50/hr/hr cloud
▾
Fits: 70B FP16 / 200B+ 4-bit / 405B with 2+ cards
Price (new): $30,000
Cloud rental: $2.50/hr (RunPod) / $8.00/hr (AWS p5)
FP16 compute: 989 TFLOPS
Memory BW: 3350 GB/s
Power: 700W TDP
3x faster than A100 for modern transformer workloads. FP8 support doubles it again.
NVIDIA H200 (141GB)
NVIDIA · 2024 · 989 TFLOPS · 4800 GB/s
141 GB
HBM3e
$40,000
$3.50/hr/hr cloud
▾
Fits: Llama 3 405B at 4-bit on ONE card. 70B FP16 with massive batch size.
Price (new): $40,000
Cloud rental: $3.50/hr (RunPod)
FP16 compute: 989 TFLOPS
Memory BW: 4800 GB/s
Power: 700W TDP
Same compute as H100 but 76% more VRAM + 43% more bandwidth. Huge win for long context.
NVIDIA B200 (192GB)Flagship
NVIDIA · 2024 · 2250 TFLOPS · 8000 GB/s
192 GB
HBM3e
$45,000
$5.00/hr/hr cloud
▾
Fits: Single card runs Llama 405B FP16 with room to spare
Price (new): $45,000
Cloud rental: $5.00/hr (limited availability)
FP16 compute: 2250 TFLOPS
Memory BW: 8000 GB/s
Power: 1000W TDP
Blackwell gen. 2.5x H100 perf. FP4 native. Current king for new deployments.
NVIDIA GB200 (NVL72)
NVIDIA · 2024 · 162000 TFLOPS · 576000 GB/s
13824 GB
HBM3e
$3,000,000
▾
Fits: Frontier training — GPT-5-scale models
Price (new): $3,000,000
FP16 compute: 162000 TFLOPS
Memory BW: 576000 GB/s
Power: 120000W TDP
Full rack: 72 Blackwell GPUs + 36 Grace CPUs. Only relevant if you're training a frontier model.
NVIDIA A10G (24GB)
NVIDIA · 2021 · 125 TFLOPS · 600 GB/s
24 GB
GDDR6
rent only
$1.00/hr/hr cloud
▾
Fits: 13-14B FP16 / 30B 4-bit
Cloud rental: $1.00/hr (AWS g5)
FP16 compute: 125 TFLOPS
Memory BW: 600 GB/s
Power: 150W TDP
AWS's workhorse inference GPU. 4x A10G matches a single A100 40GB for many workloads.
Google TPU v5p
Google · 2024 · 2765 GB/s
95 GB
VRAM
rent only
$4.20/hr/hr cloud
▾
Fits: Frontier training (Gemini-scale)
Cloud rental: $4.20/hr (GCP)
Memory BW: 2765 GB/s
Top-tier GCP training chip. Compares to H100-H200 for transformer workloads.
A100 PCIe
NVIDIA · 2026
80 GB
VRAM
rent only
▾
Fits: 80GB VRAM — large models
Auto-discovered via RunPod live pricing.
A100 SXM
NVIDIA · 2026
80 GB
VRAM
rent only
▾
Fits: 80GB VRAM — large models
Auto-discovered via RunPod live pricing.
B300
NVIDIA · 2026
288 GB
VRAM
rent only
▾
Fits: 288GB VRAM — large models
Auto-discovered via RunPod live pricing.
H100 SXM
NVIDIA · 2026
80 GB
VRAM
rent only
▾
Fits: 80GB VRAM — large models
Auto-discovered via RunPod live pricing.
H100 NVL
NVIDIA · 2026
94 GB
VRAM
rent only
▾
Fits: 94GB VRAM — large models
Auto-discovered via RunPod live pricing.
H100 PCIe
NVIDIA · 2026
80 GB
VRAM
rent only
▾
Fits: 80GB VRAM — large models
Auto-discovered via RunPod live pricing.
H200 SXM
NVIDIA · 2026
141 GB
VRAM
rent only
▾
Fits: 141GB VRAM — large models
Auto-discovered via RunPod live pricing.
NVIDIA H200 NVL
NVIDIA · 2026
143 GB
VRAM
rent only
▾
Fits: 143GB VRAM — large models
Auto-discovered via RunPod live pricing.
RTX PRO 6000 MaxQ
NVIDIA · 2026
96 GB
VRAM
rent only
▾
Fits: 96GB VRAM — large models
Auto-discovered via RunPod live pricing.
RTX PRO 6000
NVIDIA · 2026
96 GB
VRAM
rent only
▾
Fits: 96GB VRAM — large models
Auto-discovered via RunPod live pricing.
RTX PRO 6000 WK
NVIDIA · 2026
96 GB
VRAM
rent only
▾
Fits: 96GB VRAM — large models
Auto-discovered via RunPod live pricing.