Intel Gaudi 3
Intel: specifications, comparisons and alternatives.
The Intel Gaudi 3 is an AI accelerator for training and inference in data centres. According to Intel, it has 128 GB of HBM2e memory, 3.7 TB/s of bandwidth and a TDP of up to 900 W in the OAM version. For connectivity, it uses 24 ports of 200 GbE with RoCE v2.
Intel
- 128 GB
- Memory
- HBM2e
- Memory type
- 3,7 TB/s
- Memory bandwidth
- 900 W
- Power (board power or TDP)
You can see the current price and availability with the partner as soon as a link has been set up.
How are the points calculated?
Clouwino points are an arithmetic summary of the specifications the manufacturer states, on a scale of 0 to 100. It is not a benchmark and not a judgement on quality or suitability: a product with fewer points may suit your situation better. Products are compared with other products of the same type. For each property we count how far a product sits between the lowest and highest value in the group:
- FP32 compute (for: consumer), carries heavy weight
- Memory (for: consumer), carries heavy weight
- Memory bandwidth (for: consumer), carries heavy weight
- Compute units (for: consumer), carries light weight
- Boost clock (for: consumer), carries light weight
- Memory (for: professional and data centre), carries medium weight
- Memory bandwidth (for: professional and data centre), carries medium weight
- FP16/BF16 (for: professional and data centre), carries heavy weight
- FP8 (for: professional and data centre), carries medium weight
- FP64 (for: professional and data centre), carries medium weight
If too much data is missing, we do not show points. Points are relative: when a new product is added, they can shift. Read more at how we compare.
Specifications of Intel Gaudi 3
All specifications the manufacturer lists, per item. Want to see another product next to it? Pick one below.
| Specificatie | Intel Gaudi 3 |
|---|---|
| Overview | |
| Clouwino points | 33 punten |
| General | |
| Architecture | Gaudi 3 |
| Manufacturing process | TSMC 5nm |
| Form | OAM (HL-325L, OCP OAM v2.0); also PCIe card (HL-338) |
| Launch | 2024-09 |
| Compute | |
| FP32 compute | 229 TFLOPS |
| FP64 | Not stated by the manufacturer |
| FP16/BF16 | Not stated by the manufacturer |
| FP8 | 1.678 TFLOPS |
| FP4 | Not stated by the manufacturer |
| Performance notes | The Intel whitepaper does not state dense/sparse: unknown, not assumed. Matrix (MME): BF16 1,678 TFLOPS, FP8 1,678 TFLOPS, FP16 459, TF32 459, FP32 229 TFLOPS (fp32 field = MME matrix). Vector (TPC): BF16/FP16 28.7, FP8 57.3, FP32 14.3 TFLOPS. FP64 and FP4 not stated. The FP16/BF16 field has been left empty because BF16 (1,678) and FP16 (459) differ. |
| Memory | |
| Memory | 128 GB |
| Memory type | HBM2e |
| Memory bandwidth | 3,7 TB/s |
| Power and connections | |
| Power (board power or TDP) | 900 W |
| GPU interconnect | 24x 200 GbE RoCE v2; PCIe 5.0 x16 |
| Multiple instances | Not stated by the manufacturer |
Note: Figures per accelerator. TDP up to 900 W (OAM, passive or liquid); PCIe card 600 W. 96 MB on-die SRAM. 8 MME + 64 TPC. Launch date 24 Sept 2024 according to Intel press release (previously announced April 2024). The stated 24x200GbE = 4.8 Tb/s per direction.
Alternatives to Intel Gaudi 3
Products with similar specifications from the same overview. This is not a ranking and not buying advice.
- AMD Instinct MI250X128 GB · HBM2e · 3,2 TB/sCompare with Intel Gaudi 3 →
- NVIDIA H100 SXM80 GB · HBM3 · 3,35 TB/sCompare with Intel Gaudi 3 →
- NVIDIA H200 SXM141 GB · HBM3e · 4,8 TB/sCompare with Intel Gaudi 3 →
- AMD Instinct MI21064 GB · HBM2e · 1,6 TB/sCompare with Intel Gaudi 3 →
- NVIDIA A100 80GB PCIe80 GB · HBM2e · 1,94 TB/sCompare with Intel Gaudi 3 →
- AMD Instinct MI300X192 GB · HBM3 · 5,3 TB/sCompare with Intel Gaudi 3 →
Gaudi 3 as an alternative to a traditional GPU
Gaudi 3 is not a graphics GPU but an accelerator designed specifically for AI, made on TSMC 5nm. Intel supplied it as an OAM module (HL-325L, per OCP OAM v2.0) and as a PCIe card (HL-338). The press release announcing the launch is dated 24 September 2024, following an earlier announcement in April.
Inside the chip are 8 matrix engines (MME) and 64 vector engines (TPC), plus 96 MB of on-die SRAM. A general explanation of such differences can be found in what is a GPU.
The approach therefore differs from GPUs with their own interconnect standard, which affects how a cluster is built.
Because Gaudi 3 uses a different software stack from many GPUs, it is sensible to check whether your frameworks are supported by the provider.
Networking, memory and compute figures of Gaudi 3
One distinctive feature is the networking setup: 24 ports of 200 GbE with RoCE v2, together 4.8 Tb/s in each direction, alongside PCIe 5.0 x16. This lets you connect accelerators via Ethernet rather than NVLink-style links. More on memory: HBM explained.
For the matrix engine (MME), Intel states BF16 and FP8 at 1,678 TFLOPS, FP16 and TF32 at 459 TFLOPS and FP32 at 229 TFLOPS. The vector engine (TPC) is lower: FP32 at 14.3 TFLOPS. Intel does not say whether these are dense or with sparsity, so we do not assume either. FP64 and FP4 are not stated.
The matrix and vector values are separate, and it helps to check which engine each figure refers to.
What to look out for with Gaudi 3
The OAM version uses up to 900 W and can be passively or liquid cooled; the PCIe card uses 600 W. Choose the version that suits your server platform.
- Check whether your network supports 200 GbE connections.
- Only compare figures with the same units, and watch out for BF16 versus FP16.
- Look in the guide to comparing GPU clouds to see whether a provider offers this model.
Intel states a lower power for the PCIe card, which is useful if you have less cooling capacity.
You can see the current price and availability at the partner via the button on the page.
Frequently asked questions
Which versions does the Intel Gaudi 3 come in?
Gaudi 3 exists as an OAM module (HL-325L) with a TDP of up to 900 W and as a PCIe card (HL-338) with 600 W. The OAM version can be passively or liquid cooled.
Are the Gaudi 3 compute figures dense or sparse?
Intel does not state this, so we assume neither. The matrix values are BF16 and FP8 at 1,678 TFLOPS, FP16 and TF32 at 459 TFLOPS and FP32 at 229 TFLOPS.
How are Gaudi 3 accelerators connected to each other?
Via 24 ports of 200 GbE with RoCE v2, together 4.8 Tb/s in each direction. There is also a PCIe 5.0 x16 connection for the host. When setting up a cluster, that is an important point to consider.
Last updated on
