Overview with affiliate links. We receive a commission if you click through via us. Read how
Compare

NVIDIA H200 SXM

NVIDIA: specifications, comparisons and alternatives.

The NVIDIA H200 SXM is a Hopper GPU for AI training, inference and HPC, with an emphasis on memory. According to the manufacturer, it has 141 GB of HBM3e, 4.8 TB/s of memory bandwidth and a TDP of up to 700 W. The SXM version uses NVLink at 900 GB/s and supports MIG with 18 GB instances.

Illustratie van NVIDIA H200 SXM45points
Illustration, not a product photo

NVIDIA

141 GB
Memory
HBM3e
Memory type
4,8 TB/s
Memory bandwidth
700 W
Power (board power or TDP)

You can see the current price and availability with the partner as soon as a link has been set up.

How are the points calculated?

Clouwino points are an arithmetic summary of the specifications the manufacturer states, on a scale of 0 to 100. It is not a benchmark and not a judgement on quality or suitability: a product with fewer points may suit your situation better. Products are compared with other products of the same type. For each property we count how far a product sits between the lowest and highest value in the group:

  • FP32 compute (for: consumer), carries heavy weight
  • Memory (for: consumer), carries heavy weight
  • Memory bandwidth (for: consumer), carries heavy weight
  • Compute units (for: consumer), carries light weight
  • Boost clock (for: consumer), carries light weight
  • Memory (for: professional and data centre), carries medium weight
  • Memory bandwidth (for: professional and data centre), carries medium weight
  • FP16/BF16 (for: professional and data centre), carries heavy weight
  • FP8 (for: professional and data centre), carries medium weight
  • FP64 (for: professional and data centre), carries medium weight

If too much data is missing, we do not show points. Points are relative: when a new product is added, they can shift. Read more at how we compare.

Specifications of NVIDIA H200 SXM

All specifications the manufacturer lists, per item. Want to see another product next to it? Pick one below.

Specifications according to the manufacturer
SpecificatieNVIDIA H200 SXM
Overview
Clouwino points45 punten
General
ArchitectureHopper
Manufacturing processNot stated by the manufacturer
FormSXM
Launch2023-11
Compute
FP32 compute67 TFLOPS
FP6434 TFLOPS
FP16/BF161.979 TFLOPS
FP83.958 TFLOPS
FP4Not stated by the manufacturer
Performance notesFP64 34 TFLOPS (vector; Tensor Core 67) and FP32 67 TFLOPS: no mention of sparsity. FP16/BF16 1,979 and FP8 3,958 TFLOPS: WITH SPARSITY (footnote 'Values are with sparsity'); dense not on the page and not calculated. FP4 not stated.
Memory
Memory141 GB
Memory typeHBM3e
Memory bandwidth4,8 TB/s
Power and connections
Power (board power or TDP)700 W
GPU interconnectNVLink 900 GB/s; PCIe Gen5 128 GB/s
Multiple instancesMIG: up to 7 instances of 18 GB

Note: Figures per GPU (SXM column). NVIDIA marks the specifications as 'preliminary, may change'. Announced 13 Nov 2023 (SC23), available from Q2 2024. Process node not stated on the page. H200 NVL (PCIe): up to 600 W, 141 GB, 4.8 TB/s.

Alternatives to NVIDIA H200 SXM

Products with similar specifications from the same overview. This is not a ranking and not buying advice.

The H200 SXM at a glance

The H200 SXM builds on the Hopper architecture and is intended for server environments running large language models or compute-intensive simulations. NVIDIA announced it on 13 November 2023, at SC23, with availability from the second quarter of 2024.

You get it as an SXM module for use in a suitable server board. There is also an H200 NVL with a PCIe connection, with power of up to 600 W, 141 GB of memory and 4.8 TB/s. The process node is not stated on the page, so we leave it out here. Note: NVIDIA marks the specifications as preliminary and subject to change.

For those already working with the Hopper generation, the memory is the main thing to look at closely with this card.

What the H200 figures mean

The standout feature of this model is the memory: 141 GB of HBM3e fits larger models on a single GPU, and 4.8 TB/s of bandwidth keeps the compute cores fed. You can find background in what is VRAM. The power can rise to 700 W.

The compute figures are the same as those of the H100 SXM. FP64 is 34 TFLOPS and FP32 is 67 TFLOPS, with no mention of sparsity. FP16/BF16 (1,979 TFLOPS) and FP8 (3,958 TFLOPS) are stated with sparsity; the dense values are not on the page and have not been calculated. FP4 is not stated by the manufacturer. All figures given are per GPU.

Figures like these apply under standardised conditions. Your own workload may behave differently, depending on model size, batch size and software.

Choosing between H200 SXM options

Start with your server platform: an SXM module needs a board with enough power delivery and cooling for 700 W per GPU. Also check the speed of the interconnect, which is 900 GB/s with NVLink.

  • Use our checklist for GPU instances if you are looking in the cloud.
  • Only compare figures with the same units and the same sparsity label.
  • Consider whether you need MIG to share the GPU.

You can find more background on reading datasheets in GPU specifications explained.

You can see the current price and availability at the partner via the button on the page.

Frequently asked questions

What is the difference from the H100 SXM according to the data?

The main difference is the memory: 141 GB of HBM3e and 4.8 TB/s for the H200 SXM, compared with 80 GB of HBM3 and 3.35 TB/s for the H100 SXM. The stated compute figures are the same.

Are the H200 SXM specifications final?

NVIDIA marks the specifications as preliminary and says they may change. So check with the provider exactly what is being supplied before making a choice.

How many MIG instances does the H200 SXM support?

The H200 SXM supports MIG with up to 7 instances of 18 GB per GPU. This lets you split one GPU into smaller, isolated parts for different tasks.

Last updated on

Ask us
Compare Get in touch

Are you sure?