Overview with affiliate links. We receive a commission if you click through via us. Read how
Compare

NVIDIA H100 SXM

NVIDIA: specifications, comparisons and alternatives.

The NVIDIA H100 SXM is a data centre GPU from the Hopper generation, designed for AI training, inference and HPC, among other things. According to the manufacturer, it has 80 GB of HBM3 memory, 3.35 TB/s of memory bandwidth and a TDP of up to 700 W. The SXM version is connected via NVLink at 900 GB/s. That way you quickly learn about the most important characteristics.

Illustratie van NVIDIA H100 SXM40points
Illustration, not a product photo

NVIDIA

80 GB
Memory
HBM3
Memory type
3,35 TB/s
Memory bandwidth
700 W
Power (board power or TDP)

You can see the current price and availability with the partner as soon as a link has been set up.

How are the points calculated?

Clouwino points are an arithmetic summary of the specifications the manufacturer states, on a scale of 0 to 100. It is not a benchmark and not a judgement on quality or suitability: a product with fewer points may suit your situation better. Products are compared with other products of the same type. For each property we count how far a product sits between the lowest and highest value in the group:

  • FP32 compute (for: consumer), carries heavy weight
  • Memory (for: consumer), carries heavy weight
  • Memory bandwidth (for: consumer), carries heavy weight
  • Compute units (for: consumer), carries light weight
  • Boost clock (for: consumer), carries light weight
  • Memory (for: professional and data centre), carries medium weight
  • Memory bandwidth (for: professional and data centre), carries medium weight
  • FP16/BF16 (for: professional and data centre), carries heavy weight
  • FP8 (for: professional and data centre), carries medium weight
  • FP64 (for: professional and data centre), carries medium weight

If too much data is missing, we do not show points. Points are relative: when a new product is added, they can shift. Read more at how we compare.

Specifications of NVIDIA H100 SXM

All specifications the manufacturer lists, per item. Want to see another product next to it? Pick one below.

Specifications according to the manufacturer
SpecificatieNVIDIA H100 SXM
Overview
Clouwino points40 punten
General
ArchitectureHopper
Manufacturing processTSMC 4N
FormSXM
Launch2022-03
Compute
FP32 compute67 TFLOPS
FP6434 TFLOPS
FP16/BF161.979 TFLOPS
FP83.958 TFLOPS
FP4Not stated by the manufacturer
Performance notesFP64 34 TFLOPS (vector; FP64 Tensor Core 67 TFLOPS) and FP32 67 TFLOPS: no sparsity mention. FP16/BF16 1,979 TFLOPS and FP8 3,958 TFLOPS: WITH SPARSITY (NVIDIA footnote '* With sparsity'); dense values are not on the product page and have not been converted. FP4 not supported/stated. Also TF32 989 TFLOPS and INT8 3,958 TOPS with sparsity.
Memory
Memory80 GB
Memory typeHBM3
Memory bandwidth3,35 TB/s
Power and connections
Power (board power or TDP)700 W
GPU interconnectNVLink 900 GB/s; PCIe Gen5 128 GB/s
Multiple instancesMIG: up to 7 instances of 10 GB

Note: Figures per GPU (SXM column). TDP up to 700 W (configurable). Memory type, process node and launch date (announced 22 March 2022; available from Q3 2022) come from the NVIDIA press release, not the specification table. H100 NVL (PCIe) is a separate variant: 94 GB, 3.9 TB/s, 350-400 W.

Alternatives to NVIDIA H100 SXM

Products with similar specifications from the same overview. This is not a ranking and not buying advice.

What the H100 SXM is designed for

The H100 SXM is an accelerator for server environments where large AI models are trained or run, and for scientific computing tasks. NVIDIA announced the Hopper architecture in March 2022, with availability from the third quarter of that year. The chip is made on the TSMC 4N process.

The product comes as an SXM module: you do not place it as a separate card in a standard PCIe slot, but on a server board made for the purpose. The H100 NVL with a PCIe connection is a separate variant with 94 GB of memory and a bandwidth of 3.9 TB/s. The H100 SXM also supports MIG, which lets you divide one GPU into up to 7 instances of 10 GB.

Memory, power and compute precision in plain language

The memory determines how large a model or dataset can be, and the bandwidth how quickly that data reaches the compute cores. You can find more explanation of this type of memory in our article on HBM memory. The TDP of 700 W is a maximum and can be configured.

For compute precision, NVIDIA gives FP64 as 34 TFLOPS and FP32 as 67 TFLOPS, without mention of sparsity. The FP16/BF16 value of 1,979 TFLOPS and the FP8 value of 3,958 TFLOPS, however, apply with sparsity; the dense values are not on the product page. FP4 is not mentioned. If you want to know what those formats mean, read compute precision explained.

That way you keep an idea in plain words of what the card can do, without having to read through all the datasheets.

What to look out for with the H100 SXM

An SXM GPU needs a suitable server platform with enough power and cooling for 700 W per GPU. Also look at the connection between GPUs: NVLink at 900 GB/s and PCIe Gen5 at 128 GB/s play different roles, as you can read in NVLink, PCIe and GPU interconnects.

  • Only compare figures with the same units: dense with dense, sparse with sparse.
  • Check whether a provider supplies the SXM or the PCIe variant.
  • Consider whether renting via a cloud or having your own server suits you better.

You can see the current price and availability at the partner via the button on the page.

Frequently asked questions

Are the H100 SXM's FP16 and FP8 figures dense or sparse?

According to NVIDIA, the FP16/BF16 value of 1,979 TFLOPS and the FP8 value of 3,958 TFLOPS apply with sparsity. The dense values are not on the product page, and we do not convert them.

How much memory does the H100 SXM have?

The H100 SXM has 80 GB of HBM3 memory with a bandwidth of 3.35 TB/s, per GPU. With MIG you can divide the GPU into up to 7 instances of 10 GB.

Is the H100 SXM the same as the H100 NVL?

No, the H100 NVL is a separate PCIe variant. According to the data, it has 94 GB of memory, 3.9 TB/s of bandwidth and a power rating of 350 to 400 W, compared with 80 GB and up to 700 W for the SXM.

Last updated on

Ask us
Compare Get in touch

Are you sure?