The four NVIDIA data-centre GPUs most teams choose between today are the H100, H200, B200 and B300. The newest one is not always the best buy. The right choice depends on how big your model is, whether you are training or serving it, and how long the job runs.
The short answer
- H100: the reliable workhorse. Best value for fine-tuning and for serving models up to about 70B parameters at 8-bit.
- H200: an H100 with much more and faster memory. Pick it when your model or long context windows don’t fit comfortably in 80 GB.
- B200: the Blackwell generation. Choose it for large training runs and high-volume inference where finishing faster saves money.
- B300: Blackwell Ultra with 288 GB per GPU. For the largest models, long reasoning workloads and when you want the fewest GPUs per model.
Side by side
| GPU | Generation | Memory per GPU | Memory bandwidth | Low-precision formats | zaryacloud on-demand |
|---|---|---|---|---|---|
| H100 | Hopper | 80 GB HBM3 | about 3.35 TB/s | FP8 | |
| H200 | Hopper | 141 GB HBM3e | about 4.8 TB/s | FP8 | |
| B200 | Blackwell | about 180 GB HBM3e | about 8 TB/s | FP8, FP4 | |
| B300 | Blackwell Ultra | 288 GB HBM3e | about 8 TB/s | FP8, FP4 |
Memory and bandwidth are NVIDIA’s published figures for the SXM / HGX versions. Prices are per GPU per hour and update automatically from our price list.
Why memory matters more than you think
For most AI work, the first question is not “which GPU is fastest?” but “how many GPUs do I need just to fit the model?”. A 70-billion-parameter model needs about 140 GB at 16-bit precision, before you count the attention cache. That is two H100s, one H200 is borderline, and one B300 fits it easily. Fewer GPUs means less splitting work across cards, simpler setup and often a lower bill.
Memory bandwidth decides how fast a GPU can read those weights. When serving a chatbot, every generated word reads the whole model from memory, so the H200’s and Blackwell’s faster memory turns directly into more words per second.
When the H100 is enough
The H100 is mature, widely supported by every framework, and the cheapest of the four per hour. It is the right default for fine-tuning models up to about 30B parameters, LoRA fine-tuning of 70B models on a single 8-GPU server, and serving 8B to 70B models at 8-bit precision.
When to pay for the H200
The H200 uses the same compute as the H100 but adds 76% more memory and faster bandwidth. It shines when your model only just fails to fit on H100s, when you need long context windows (32K tokens and up), or when you want to serve a 70B model on one GPU instead of two.
When Blackwell (B200 and B300) saves money
Blackwell GPUs cost more per hour but often finish jobs much faster. In many training and inference workloads, teams see a B200 complete the same job in roughly half the time of an H100, depending on the model and software. If a training run takes 10 days on H100s and 5 on B200s, the B200 can be cheaper overall even at a higher hourly price. Blackwell also adds FP4, which lets very large models run in less memory with good quality for many tasks.
Pick the B300 when you work with the largest open models (400B to 700B parameters), long reasoning chains, or simply want the fewest GPUs per model.
A quick rule of thumb
- Experimenting or on a budget: H100, or spot instances.
- Serving a 70B model in production: H200 or B200.
- Training something big on a deadline: B200 or B300 cluster, reserved for the length of the project.
- Running the largest open models: B300.