Overview with affiliate links. We receive a commission if you click through via us. Read how
Compare

Calculating AI infrastructure costs: GPUs, storage and network

A calculation model for AI costs: GPU hours, storage, network traffic, management and the costs that are often forgotten.

Door de redactie van ClouwinoGepubliceerd Laatst bijgewerkt

Short answer

Add up the costs of AI in five groups: compute (GPU hours), storage, network traffic, software and management, and the cost of idle time or trial runs. For each group, calculate usage times the rates you have checked yourself with providers, and allow a margin for unexpected use. Then measure the actual cost per request.

Who is it for: Teams and budget holders who want to estimate the costs of an AI project.

The five cost items

A calculation model in steps

Start with the workload: how many GPU hours in a month, what amount of data and what traffic. Multiply each by the provider’s rate and add up. Allow a generous margin for trial runs and unexpected use, unless your own measurements justify a different margin. Use our cost calculator to compare options side by side.

Cost per request

For inference, a useful measure is the cost per request or per thousand requests. Divide the monthly costs by the number of requests. This shows you whether an application is economically viable and what smaller models or more efficient hardware would deliver.

Forgotten costs

How to limit costs

Cost items for AI

Fill in the rates yourself with checked amounts.
ItemWhat you measurePitfall
GPU hoursNumber of GPUs times hoursIdle machines
StorageAmount of data times timeOld copies and checkpoints
NetworkOutgoing and between locationsMoving large datasets
ManagementYour team’s hours and softwareUnderestimated in plans
Trial runsFailed and experimental runsNot included in the budget

Who is this less suitable for?

Frequently asked questions

How much margin should I allow?

A generous margin is a common starting point for a budget. Adjust it using your own measurements.

What is the biggest surprise?

Often the sum of small items: storage, traffic and forgotten machines.

How do I compare renting and buying?

Calculate both over the same period and with the same hours of use. See the guide on renting or buying GPUs.

Ask us
Compare Get in touch

Are you sure?