Overview with affiliate links. We receive a commission if you click through via us. Read how
Compare

Compute Precision Explained: FP32, FP16, FP8 and FP4 in AI

What FP64, FP32, FP16, FP8 and FP4 mean, and how to read GPU performance figures.

Blog5 min read

Short answer

FP stands for floating point, the number of bits a GPU uses to represent a number: 64, 32, 16, 8 or 4. Fewer bits means less memory and usually more operations per second, but also less accuracy. AI training and inference increasingly use lower precisions. With performance figures, pay attention to the precision and to whether a figure is “dense” or “with sparsity”.

What do the numbers mean?

Compute precisions in GPUs and AI (general description)
PrecisionTypical useCharacteristic
FP64Scientific simulationsVery accurate, uses a lot of memory and compute time
FP32General compute, graphics, some trainingStandard for a lot of software
FP16 and BF16Training and inference of AI modelsLess memory, often faster
FP8Training and inference where the model allows itEven less memory, extra attention needed for accuracy
FP4Inference with heavily compressed modelsVery compact, only supported on newer hardware

Dense or sparse?

Some manufacturers quote performance twice: as “dense” (ordinary calculations) and “with sparsity” (where some of the values may be zero and are skipped). The sparse figure is higher, but only counts if your model and software are suited to it. So compare dense with dense.

What does this mean for your choice?

Check which precision your software uses and compare GPUs at that precision. A card that excels at FP4 will not help you if your work needs FP32. On our professional and data centre GPUs page, each data centre GPU lists its precisions and a note on the manufacturer’s figures.

A practical example

A team running an existing model in 16 bits looks at a GPU’s FP16 or BF16 figures. A team running a compressed model in 8 or 4 bits looks at FP8 or FP4. Scientific software that demands a lot of accuracy looks at FP64. One card may be strong at one precision and less so at another.

That is why, for data centre GPUs, we give the manufacturer’s figure per precision, with a note when sparsity is involved. An empty cell means the manufacturer has not published a figure.

How much memory does an AI model take in FP16, FP8 or FP4?

The precision determines how much space each number takes up in memory. You can work this out yourself: number of parameters × number of bytes per parameter. A byte is 8 bits. The weights of the model alone then take up this much space:

PrecisionBytes per valueMemory for N billion parameters (weights only)
FP324roughly 4 × N GB
FP16 or BF162roughly 2 × N GB
FP81roughly N GB
FP40.5 (slightly more because of scale factors)roughly 0.5 × N GB or slightly more

For N, fill in the number of billions of parameters of your model. This is a lower limit. While running, you need extra memory for intermediate results and the context of the conversation, and that differs per software and setting.

FP4 often comes with a scale factor for each small group of numbers. For its NVFP4 format, NVIDIA describes an FP8 scale factor per 16 values, which comes to 4.5 bits per value. That shows that “4 bit” can in practice cost slightly more memory. The GPU’s memory is listed in the specifications; see GPU specifications explained and what is HBM memory.

What is the difference between FP16 and BF16?

Both use 16 bits, but they divide them differently. A number consists of a sign, an exponent (the range: how large or small) and a mantissa (the precision: how many digits).

FormatExponent bitsMantissa bits
FP32823
BF1687
FP16510

Google describes bfloat16 as a format with eight exponent bits and seven mantissa bits, with the same range as FP32 but less precision. FP16 has a smaller range but more mantissa bits. Which of the two suits you depends on the software and the model; many frameworks let you choose.

FP4 also has its own structure. NVIDIA calls its 4-bit format E2M1: one sign bit, two exponent bits and one mantissa bit. That is why it can only represent a limited number of different values, and scale factors are needed to cover the range of the data. With FP8 and FP16 there is more room; with FP4 every bit is precious.

How do you check for yourself whether a GPU supports your precision?

  1. Find out which precision your model or software uses. This is usually in the documentation or on the model page.
  2. Open the manufacturer’s datasheet and find the table with performance per precision.
  3. Read the footnotes: if it says “with sparsity”, do not compare it with a dense figure from another card.
  4. Check in your framework’s documentation whether that precision is supported on this type of GPU.
  5. Also look at memory and memory bandwidth. A high compute figure helps little if the model does not fit.
  6. Check whether the model is available in the precision you want. Some models are offered in several versions; the model page usually says so.
  7. First test with a small part of your work whether the results stay good enough for your application, because lower precision can affect quality.

You can compare different cards on the GPU page; see also graphics card or data centre GPU.

Frequently asked questions

Is lower precision worse?

It is less accurate, but for a lot of AI work that is sufficient. Whether it works in your case depends on the model.

Are TOPS and TFLOPS the same?

No: TOPS often counts operations on integers, TFLOPS floating-point operations.

How do I calculate how much memory an AI model needs?

Multiply the number of parameters by the number of bytes per parameter: 4 for FP32, 2 for FP16 or BF16, 1 for FP8 and about 0.5 for FP4. Add memory for intermediate results and context.

Further reading

Last updated on

Ask us
Compare Get in touch

Are you sure?