Workload guide · AI
Local AI inference: hardware requirements guide
Running LLMs or other models locally (Ollama, vLLM, llama.cpp).
Hardware requirements by tier
Three tiers for every workload: the minimum that works, the recommended sweet spot, and the comfortable headroom level. These are the same tiers the WisePC decision engine uses when it plans a build around your goal.
| Tier | CPU cores | RAM | Storage | GPU | Network |
|---|---|---|---|---|---|
| Minimum | 8 cores | 16 GB | 1 TB (ssd) | ai | 1 GbE |
| Recommended | 12 cores | 32 GB | 1 TB (ssd) | ai | 2.5 GbE |
| Comfortable | 16 cores | 64 GB | 2 TB (ssd) | ai | 10 GbE |
Which tier do you need?
- Pick Minimum (8 cores, 16 GB RAM) only for a single-purpose machine on a tight budget — expect little headroom.
- Recommended (12 cores, 32 GB RAM) is the sweet spot: enough for the workload plus the usual side-services, without overspending.
- Pick Comfortable (16 cores, 64 GB RAM) when this workload shares the machine with others or will grow — you pay for headroom, not for anxiety.
- GPU rule for this workload: ai.
Frequently asked questions
How much RAM does local ai inference need?
16 GB is the sensible minimum, 32 GB covers most real setups, and 64 GB gives comfortable headroom for growth and extra services.
How many CPU cores does local ai inference need?
A 8-core CPU is the minimum, 12 cores is the recommended sweet spot, and 16 cores is comfortable when it shares the machine with other workloads.
Does local ai inference need a dedicated GPU?
A dedicated GPU is strongly recommended — this workload does AI compute.
What storage and network does local ai inference expect?
Storage: 1 TB of SSD is the recommended baseline (1 TB minimum, 2 TB comfortable). Network: 2.5 GbE is the recommended baseline.
What runs well alongside local ai inference?
It pairs naturally with: Image generation, AI experimentation, Databases.