Deploy guide
Deploy Ollama on Vast.ai
Run an Ollama server on a rented GPU and reach it over HTTPS, with automatic model pulls on boot, health checks and instance replacement. Cheapest card that fits 7B–13B models, 4-bit quantised: RTX A4000 (16 GB) at $0.09/hr — about $66/month if it ran 24/7.
Quickstart
| Image | ollama/ollama:latest |
|---|---|
| Container port | 11434 |
| Health endpoint | / |
| Minimum VRAM | 16 GB |
- Create a deployment with the ollama/ollama:latest image. In the dashboard choose New deployment, paste ollama/ollama:latest, and set the container port to 11434.
- Pick the GPU pool. Select SKUs with at least 16 GB of VRAM — the cheapest right now is RTX A4000 (16 GB) at $0.09/hr.
- Set the health endpoint and start. Use / as the health endpoint so a worker only receives traffic once the container answers. Start the deployment and open the public URL it gets.
Things that bite
- The server listens on 11434 and answers / as soon as it is up — model weights are pulled lazily on the first request, so warm the model you care about before pointing traffic at it.
- Add OLLAMA_KEEP_ALIVE so the model stays resident; otherwise the first request after idle pays the load time again.
- Quantised models are small: a 4-bit 7B fits in 8–12 GB, but 16 GB leaves room for context and a second model. Use 24 GB for 13B and above.
What it costs
GPU time is Vast.ai's own price, billed per second to your Vast.ai account; CUDAnetes adds a flat subscription (free plan or $19/mo on Pro) and takes no markup. A fleet running 24/7 for a month costs $66 per card at the cheapest live offer, before the subscription.
See the live GPU price index for current rates, or the Vast.ai vs RunPod comparison for the fixed-price alternative.
Questions
Is Ollama on Vast.ai cheaper than a hosted API?
For steady traffic, yes: one consumer card at the low end of the Vast.ai market costs less per hour than per-token pricing at scale. For occasional requests, hosted APIs are usually cheaper — GPU rent runs while the deployment is up.
How do I pull a model at boot?
Put the pull in the container command or an entrypoint script your image runs, so a fresh worker is ready before it passes the health check.
Can I use my own Ollama image?
Yes — any Docker image that serves on a port and answers a health path works; the port and health endpoint are deployment settings, not build-time choices.
Deploy it now
Free plan: one deployment with up to five GPUs, no credit card. Any Docker image, one public URL.