What cloud infrastructure do you need for AI?
Which components an AI application needs: computing power, storage, network and management, and how to link them to your goal.
An AI application needs four building blocks: computing power (often GPUs), fast storage for data and models, a network that delivers data fast enough, and management for security and costs. What exactly you need depends on whether you train, fine-tune or only use a model (inference), and on the size of your model and data.
Who is it for: Teams and buyers planning an AI project who want to know what infrastructure goes with it.
Start with your goal
The infrastructure follows from what you want to do. Using an existing model via a service requires almost nothing of your own. Fine-tuning a model with your own data temporarily requires a lot of computing power. Training your own model from scratch requires the most resources and is not necessary for most organisations. So first decide which of these three situations applies to you.
The four building blocks
- Computing power: GPUs or other accelerators that carry out many calculations at once. For small applications, an ordinary processor may be enough.
- Storage: room for datasets, intermediate results and models. Object storage is often suitable for large amounts of data.
- Network: data has to reach the computing power fast enough. When training on several machines, the connection between those machines also matters.
- Management: access rights, encryption, logging and cost monitoring.
Is your team training, fine-tuning or using a model?
Training requires a lot of computing power for a limited period and a lot of data. Fine-tuning requires less, but still powerful hardware with enough memory. Inference, using a trained model, requires less per request but runs continuously and has to be fast and reliable.
Choose where it runs
You can run AI work in a public cloud, with a specialised provider, on your own hardware or in a mix. Public options give quick access and scalability. Your own hardware gives more control over data and can be more economical with constant usage, but requires investment and management. See also the guide on private or public AI.
Watch costs and availability
Powerful GPUs are not necessarily available straight away and cost a lot per hour. Plan capacity in advance, measure usage and switch off environments you are not using. The guides on costs and capacity go into this in more depth.
From goal to infrastructure
| Goal | Computing power | Storage and network |
|---|---|---|
| Using a model via a service | Hardly any computing power of your own | Limited; watch where data goes |
| Fine-tuning a model with your own data | Temporarily powerful GPUs with enough memory | Fast storage for datasets and intermediate results |
| Running a model in production | Constant capacity, matched to the number of requests | Low latency, reliable network |
| Training your own model | Many GPUs that communicate quickly with each other | Very large datasets, fast shared storage |
Who is it not ideal for?
- If you only use a ready-made AI service: the guide on AI data and privacy is then mainly relevant.
- If you are running a single experiment: a short, temporary environment is then enough.
Frequently asked questions
Do I absolutely need a GPU?
No, not in every case. Small models and simple applications sometimes run on an ordinary processor. For training and large models, a GPU is practically necessary.
Is the cloud or your own hardware better for AI?
That depends on usage, data and budget. The cloud suits fluctuating demand; your own hardware can be interesting with constant usage.
How much storage does AI need?
That depends on your data and models. Allow generously, because datasets, copies and intermediate results grow quickly.
