Overview with affiliate links. We receive a commission if you click through via us. Read how
Compare

What cloud infrastructure do you need for AI?

Which components an AI application needs: computing power, storage, network and management, and how to link them to your goal.

Door de redactie van ClouwinoGepubliceerd Laatst bijgewerkt

Short answer

An AI application needs four building blocks: computing power (often GPUs), fast storage for data and models, a network that delivers data fast enough, and management for security and costs. What exactly you need depends on whether you train, fine-tune or only use a model (inference), and on the size of your model and data.

Who is it for: Teams and buyers planning an AI project who want to know what infrastructure goes with it.

Start with your goal

The infrastructure follows from what you want to do. Using an existing model via a service requires almost nothing of your own. Fine-tuning a model with your own data temporarily requires a lot of computing power. Training your own model from scratch requires the most resources and is not necessary for most organisations. So first decide which of these three situations applies to you.

The four building blocks

Is your team training, fine-tuning or using a model?

Training requires a lot of computing power for a limited period and a lot of data. Fine-tuning requires less, but still powerful hardware with enough memory. Inference, using a trained model, requires less per request but runs continuously and has to be fast and reliable.

Choose where it runs

You can run AI work in a public cloud, with a specialised provider, on your own hardware or in a mix. Public options give quick access and scalability. Your own hardware gives more control over data and can be more economical with constant usage, but requires investment and management. See also the guide on private or public AI.

Watch costs and availability

Powerful GPUs are not necessarily available straight away and cost a lot per hour. Plan capacity in advance, measure usage and switch off environments you are not using. The guides on costs and capacity go into this in more depth.

From goal to infrastructure

A general indication; the actual requirements depend on the model and data.
GoalComputing powerStorage and network
Using a model via a serviceHardly any computing power of your ownLimited; watch where data goes
Fine-tuning a model with your own dataTemporarily powerful GPUs with enough memoryFast storage for datasets and intermediate results
Running a model in productionConstant capacity, matched to the number of requestsLow latency, reliable network
Training your own modelMany GPUs that communicate quickly with each otherVery large datasets, fast shared storage

Who is it not ideal for?

Frequently asked questions

Do I absolutely need a GPU?

No, not in every case. Small models and simple applications sometimes run on an ordinary processor. For training and large models, a GPU is practically necessary.

Is the cloud or your own hardware better for AI?

That depends on usage, data and budget. The cloud suits fluctuating demand; your own hardware can be interesting with constant usage.

How much storage does AI need?

That depends on your data and models. Allow generously, because datasets, copies and intermediate results grow quickly.

Ask us
Compare Get in touch

Are you sure?