GPU TCO Analysis: The Economics of Scale
What is the TCO of an NVIDIA H100 cluster?
The Total Cost of Ownership (TCO) for an 8-GPU H100 node over 3 years ranges from $450k to $650k. This includes CapEx (hardware purchase), OpEx (power at $0.06-0.12/kWh), and colocation fees ($150-250/kW/month). Buying typically breaks even against cloud leasing at 14-18 months of high utilization.
Key Data Points
- H100 Node CapEx: ~$300,000 - $350,000
- Power Density: 10.2kW per 8-GPU node (max load)
- Break-even Point: 14-18 months vs CSP prices
- Residual Value: 25-35% after 36 months
- OpEx Load: ~40% of lifetime total cost
Hardware Ownership Models
Cloud (On-Demand)
Fastest deployment, zero CapEx, but highest $/hour and zero residual value.
- × $3.00 - $4.50 / GPU hour
- ✓ No long-term liability
Leasing (Operating)
Off-balance sheet, fixed payments, predictable OpEx without large upfront cash outlay.
- ✓ Predictable cash flow
- × Total cost > Purchase
Direct Purchase
Lowest effective hourly rate over 3+ years and retains asset residual value.
- ✓ ~$1.20 effective hour rate
- × High initial CapEx
Frequently Asked Questions
What are the hidden costs of GPU ownership?
Network transceivers, cabling, top-of-rack switches, and specialized insurance are often overlooked, adding 10-15% to the initial hardware CapEx.
How does cooling type affect TCO?
Liquid cooling has higher initial setup costs (CapEx) but reduces power consumption (OpEx) by 10-15% and increases hardware lifespan by maintaining lower operating temperatures.
Is it better to lease or buy B200s?
For first-generation Blackwell (B200) deployments, many operators prefer 12-24 month leases to mitigate the risk of rapid engineering iterations, despite the higher effective cost.
Explore More
Related Tools
GLRI (GPU Lease Rate Index)
Track H100/A100/B200 lease rate trends - core market data
Open Tool