Choosing a GPU instance: a checklist for your provider
A checklist of questions for your provider before you choose a GPU instance in the cloud.
For a GPU instance, check at least eight points: card type and memory, number of GPUs, availability in your region, storage and network, data location, payment model, support and exit options. Ask every provider the same questions so that you can compare the answers, and test with your own work before you scale up.
Who is it for: Teams who want to rent a GPU instance from a cloud provider.
Start with your requirements
Write down what you want to do, with how much data, how often and how fast. Without those requirements, a comparison of instances is not meaningful. Also read the guide on capacity planning.
The checklist
- Card and memory: which type of GPU and how much VRAM per card?
- Number of GPUs: can you get one or several per machine?
- Availability: can the capacity be obtained straight away or do you have to wait?
- Region and data location: where does it run and where is the data stored?
- Storage and network: how fast does data reach the GPU?
- Payment: per hour or through a contract, and what happens to storage when you stop?
- Support: what does the provider do when there are problems?
- Exiting: how do you take your data and models away again?
Test before you scale up
Run a representative part of your work on a small instance. Measure turnaround time, GPU usage and how much it costs. See whether the data supply is the limiting factor.
Privacy and agreements
If you process personal data or confidential data, arrange the agreements: data processing agreement, location of processing and retention periods. See the guide on AI, data and privacy.
Keep a way out
Hardware and offerings change quickly. Store models and data in common formats so that you can switch. See the guide on a cloud exit strategy. Then also look at how your data and models are stored and whether you can export them yourself in a common format, so that a later switch does not get stuck on a provider’s own file format.
Questions and why they matter
| Question | Why | Where do you find it? |
|---|---|---|
| How much memory per card? | Determines the model size | Provider documentation |
| Is the capacity available? | Waiting time costs time | Provider or status page |
| Where is the data stored? | Legislation and latency | Terms and documentation |
| How fast is the storage? | Prevents the GPU from sitting idle | Documentation, your own test |
| How do I exit? | Prevents lock-in | Terms |
Who is it not ideal for?
- Anyone who only uses an online AI service.
Frequently asked questions
Should I take several GPUs straight away?
Start with what you need and scale up if it turns out to be necessary.
Can I test with a trial period?
That differs per provider; ask about it.
Where do I compare providers?
Use our comparison tool and the guide on comparing GPU cloud.
