Documentation

Overview

What it does

CUDAnetes runs your Docker container on GPUs rented from Vast.ai. You describe a deployment — an image, a GPU type and a fleet size — and it rents the machines, keeps them healthy, balances requests across them, scales with traffic and moves to cheaper GPUs when prices drop. A deployment is the object you create, start, pause and delete.

How you connect

Each deployment gets one stable public HTTPS endpoint. You send HTTP requests to it; CUDAnetes does not provide an SSH or tunnel endpoint and is built for inference APIs and model endpoints, not interactive sessions.

What it costs

You bring your own Vast.ai API key, so you pay Vast.ai directly — CUDAnetes adds no GPU markup. Every feature is included on every plan; a subscription only lifts the Free plan's deployment, GPU, and request limits (1 deployment, 5 GPUs per deployment, 120 requests/min).

Quickstart

From zero to a public URL in a few minutes.

If you enabled proxy auth, send Authorization: Bearer <authKey>.

Billing

Costs come from two independent places:

GPU rental

Billed by Vast.ai, with no markup. You attach your own Vast.ai API key, so instances are rented on and billed to your Vast.ai account directly. CUDAnetes never resells GPU time or charges your Vast balance. Vast.ai is prepaid, so keep your balance funded — top-ups and auto-billing are handled in your Vast.ai account. At zero balance Vast.ai stops instances and storage keeps billing; with a saved card it can auto-bill, without one it may delete instances and data after a grace period. See Vast.ai billing and the balance FAQ.

Orchestration

A CUDAnetes subscription (optional — Free costs nothing). All features are included on every plan; tiers differ only in limits and support. See Pricing for the current plans.

What is billed depends on the deployment state:

Pausing stops GPU billing but not storage. Storage is normally a small fraction of GPU time. Only Stop (or Delete) destroys the disks and ends storage charges.

Upgrade from Pricing or the dashboard; the billing portal (dashboard account menu) manages payment, invoices and cancellation. Cancelling keeps the plan active until the end of the paid period; a payment failure that revokes it drops to Free immediately — new deployments then follow Free limits and fleets above the Free GPU cap are scaled down.

Cap GPU pricing with maxPricePerHour: CUDAnetes only rents offers at or below that hourly price — the cap covers new rentals, not storage or bandwidth. For example, 3 instances at $0.50/h each is a $36/day GPU maximum; actual costs are often lower. GPU spend and Vast.ai invoices are in the Vast.ai console.

Cap Vast bandwidth pricing with maxBandwidthCostPerTb (default $10/TB): CUDAnetes only rents hosts whose upload and download price is at or below that rate. Vast hosts set their own rates and the cheapest GPU often carries the most expensive bandwidth; 0 disables the cap. The cap bounds the rate, not the total — actual bytes are still billed by Vast. See Advanced host options for the supply tradeoff.

Vast.ai costs & reliability

Vast.ai bills three independent meters. CUDAnetes does not resell any of them; you attach your own key and pay Vast.ai directly.

The GPU $/hr shown in the marketplace (and Vast's dph_total) covers GPU and disk, not network. A host can look cheap per hour and still produce a large bill.

Bandwidth can dominate the bill

Vast charges for upload and download separately, with no free allowance. Image and model pulls are download; inference responses are upload; health checks and proxy traffic count too. The cost of transferring X GB at a rate of $Y/TB is (X / 1000) × Y.

Example: a single 30 GB model pulled at $39/TB costs about $1.17. At a $0.13/hr rental, that one-time download can be a large share of the first hour. Users have reported bandwidth accounting for roughly 70% of a one-hour rental after downloading a 30 GB model — a community report, not a CUDAnetes measurement.

Hosts set their own rates. Across the single-GPU marketplace we have seen everything from $0/TB to $66.67/TB, and the cheapest GPU offers are often the most expensive for traffic. CUDAnetes caps the rate at $10/TB by default; see Advanced host options for the knob and its supply tradeoff.

Redundancy beats chasing a perfect host

Vast.ai is a marketplace of independent hosts, from datacenters to consumer rigs. A cheaper offer is not automatically a worse one. For availability, several workers matter more than the host badge: CUDAnetes health-checks every worker, replaces unhealthy ones, blacklists failing machines and spreads a fleet so one host failure takes down at most one worker. Three cheap, unverified hosts can be a better bet than a single “reliable” one.

Filters such as verified-only, datacenter-only and a reliability floor are optional levers: they shrink the pool and can cost more, so use them for a specific requirement, not as a prerequisite for uptime. A replacement can take several minutes and sometimes longer; paused workers keep their disks (storage billing continues) but cannot serve traffic. A managed GPU provider may be more consistent at a higher price; Vast is cheaper and more variable by design. CUDAnetes does not control host quality and does not provide an uptime SLA.

Rule of thumb: cap the bandwidth rate and run a spare worker if an outage costs more than the spare. The cap bounds the rate, not the total: 100 TB at $10/TB is still $1000.

Deployment and worker states

There are two levels. A deployment is your fleet — the image, GPU pool and desired size you start, pause and stop. A worker is one Vast.ai instance rented for that deployment. Deployment status is what you asked for; worker status is what one instance is doing. They are independent: running does not mean the GPUs are ready, and a running deployment can have zero workers.

Deployment states

Pause vs Stop. Pause stops the fleet but keeps the instances and their disks; Start resumes the same containers, so startup is fast and data on disk survives — storage keeps billing. Stop destroys the instances and disks and ends storage billing; the deployment and its settings stay, and Start rents fresh instances and pulls the image again.

Retries. If a pause or stop fails partway, the deployment stays pausing/stopping; the same action retries it.

Editing a deployment

Settings are editable only when stopped. Stop destroys the instances and disks; Paused keeps them but is frozen. After saving, Start rents a fleet with the new settings.

GPU workers

A GPU worker is one Vast.ai instance rented for a deployment and labeled cudanetes-<deploymentId>. The statuses below are CUDAnetes's own, not Vast's actual_status: Vast reports an instance as running while a worker is still loading — the GPU is already billed, but your container is not serving yet.

Pausing vs destroying. Pausing a deployment leaves its workers paused with their disks kept on Vast.ai. Closing a schedule window destroys workers outright — no paused instances, no disks.

Fleet health on the dashboard is derived from worker counts: no GPUs when none are ready or starting, degraded when some are ready and some unhealthy, otherwise ok.

Container requirements

CUDAnetes sends your image and container options to Vast.ai, which starts the container; CUDAnetes then proxies HTTP traffic to one published port.

Image. Use a Docker image reference, not a web URL: nginx:latest, ghcr.io/acme/api:1.2.3, or quay.io/acme/api:latest. CUDAnetes passes it to Vast.ai unchanged; Vast pulls it on the GPU host, so the registry must be reachable from that host. See Vast.ai Docker environment. For a private registry, set both fields:

Image:        ghcr.io/acme/private-api:1.2.3
Image Login:  -u acme-bot -p ghp_xxxxxxxx ghcr.io

Image Login takes the arguments you would pass to docker login; the last value is the registry host (docker.io, ghcr.io, …). Use a registry access token, not your account password.

Vast.ai templates are not in the MVP — we are researching the best integration and asking users whether they need it. For now, use the image from a template as Image. If templates matter to you, tell us at [email protected].

Port. Choose one HTTP port CUDAnetes should route, and set it as Container Port — for example, 8000. Inside the container, listen on 0.0.0.0:<port>, not 127.0.0.1; otherwise Vast.ai cannot reach the app. CUDAnetes asks Vast.ai to publish that port. Vast.ai assigns the public port, and CUDAnetes uses it for health checks and API traffic — you never configure the public port yourself. CUDAnetes routes one port per deployment. Other ports are not proxied. Multiple ports may be supported later. See Vast.ai networking & ports.

Environment variables (dockerEnv). Key/value pairs passed into the container process, like docker run -e KEY=VALUE. Enter one KEY=VALUE per line. Use plain key names — options starting with - belong in Docker flags. Values are stored in plain text and visible in deployment settings, so use this for non-secret configuration only. Example: HF_HOME=/data/huggingface.

Docker flags (dockerArgs). Extra options for how Docker starts the container, one per line. Write one option per line as --flag=value (--shm-size=1g, --ipc=host), not --flag value; blank lines are ignored. They change how Docker runs the container, not the arguments your app receives — those belong in the image's CMD/ENTRYPOINT or in Container args below. See Docker run options. Use Container Port for the app port. CUDAnetes creates the port mapping the proxy connects to, so you don't need -p/--publish here. If you add one, Vast.ai may open an extra port, but CUDAnetes won't route it.

Container args (containerArgs). Arguments passed to the image's ENTRYPOINT, one per line; each line is one argument, with no shell expansion or quoting. Use these for the server's own flags, e.g. vLLM's --model and its value. Leave empty to keep the image's default CMD. These are separate from dockerArgs, which only change how Docker starts the container.

Auth (optional). Add a bearer token and enable proxy auth. The proxy then accepts only requests with Authorization: Bearer <authKey> and returns 401 for everything else.

Health check

Why. Vast.ai hosts are operated by different providers, and running only means the container started — not that the host or your app works. Health checks let CUDAnetes spot broken workers quickly, keep traffic on healthy ones, and replace failures automatically.

What. Docker has no HTTP health endpoint by default; your image must expose one. CUDAnetes calls GET <healthEndpoint> every 5 seconds, with a 5-second timeout and no auth headers; any 2xx means the worker is ready. vLLM, TGI, llama.cpp, and TEI use /health; Triton uses /v2/health/ready; TorchServe uses /ping. A custom FastAPI/Flask/Express app should add /health; nginx or static files should return 200 there (or proxy it to the backend). Default is /health; change it in deployment settings. An image without a readiness endpoint never becomes ready and is replaced.

How. Return 503 while the app is starting or loading a model, and 200 only when it can serve requests. If GPU initialization matters, check it at startup and keep returning 503 until it succeeds. Health is not a GPU benchmark — it only verifies what your endpoint checks. Health checks are requests from CUDAnetes to the worker; client requests are what the proxy routes.

The 10-minute limits are per phase, not total; a Vast crash or eviction is handled sooner.

Choosing a GPU

Available GPU types depend on Vast.ai marketplace availability. Common options:

Each GPU is tagged with a performance tier — Entry, Standard, Advanced, Pro, or Elite — based on its compute throughput (dlperf). Use the tier badge in the GPU picker to compare cards at a glance.

Each rentable GPU is a SKU (stock keeping unit) — a model and VRAM pair such as RTX_4090:24; the same card at different VRAM sizes is a different SKU. A deployment's pool is the set of SKUs it is allowed to run on.

Availability and prices are live — the GPU picker and GET /v1/gpu list the SKUs currently rentable, with their current minimum price. Set maxPricePerHour and minReliability to filter Vast.ai instances: higher reliability means fewer interruptions but higher cost. GPU price is not the whole bill — see Vast.ai costs & reliability.

Current rates per card, refreshed hourly, are in the GPU price index; stack-specific walkthroughs live in the deploy guides (vLLM, Ollama, ComfyUI).

Advanced host options

These options narrow which Vast.ai offers can be rented. Tighter filters mean fewer hosts and longer waits; they do not raise the GPU price cap. maxPricePerHour and minReliability are covered in Choosing a GPU.

Max bandwidth cost

One shared cap for upload and download, configured in $/TB. Presets are No limit, $1, $2.50, $5, $10, $20 and $40; 0 disables the cap. A host is eligible only when both its upload and download rates are at or below the cap. The cap applies to new rentals and optimizer searches; it does not reprice or evict a running worker.

It is a rate cap, not a spending budget. At $10/TB, 100 TB of traffic still costs $1000. The actual rates of the chosen host are written to the gpu-worker-provisioned event, for example $3.91/TB up / $2.60/TB down.

Snapshot from the single-GPU marketplace, October 2026. At $1/TB A100 and H100 disappear and only three models keep at least five offers. At $5–$10/TB, B300, RTX PRO 4500 and RTX PRO 6000 Max-Q can disappear. Keep $10/TB for general use; go lower only if cost matters more than supply, and raise it deliberately for those cards.

The optimizer respects the same rule: it never moves a worker to a host with a higher upload or download rate than the current one. If the current worker has no stored rate snapshot (for example, an instance adopted after a restart), it does not migrate. Manager worker rows show the rental-time rates, and the gpu-worker-provisioned event records them.

Scaling

CUDAnetes manages the GPU workers of a deployment as one group, called a fleet. Each worker is one rented GPU. The scaling policy decides how many workers the fleet has:

manual

The default. You say how many GPUs you need and CUDAnetes rents or destroys workers until the fleet matches. A new deployment starts at Min GPUs; change the count with the −/+ control on the deployment page. Use it when you know the capacity you need or when load is hard to read from requests (one long job per GPU). Scaling also works while paused or stopped — a stopped deployment starts with the new count.

requests

For request-driven services. The fleet follows traffic. Use it when each unit of work is a separate request and concurrency tracks GPU load: an HTTP API, chat/LLM inference, image generation, or a web backend. It watches requests in flight, not GPU utilization — a single long-running job per GPU gives it little to react to. Each window has two thresholds, scaleUp and scaleDown, measured in concurrent requests per GPU: 30 targets one GPU per 30 requests in flight, and scale-down must stay below scale-up. Recent 503 rejections (no ready worker, or all at Max requests per GPU) count as extra demand and hold scale-down back. CUDAnetes takes the highest target across windows, so a short spike adds GPUs quickly while longer windows keep them from being removed too soon. The default windows look back 2, 10 and 30 minutes; they are not fixed — add, remove or edit windows and thresholds in the form or scalingConfig.windows.

Min and Max GPUs bound both policies. Min GPUs is the floor the manager aims to keep; 0 lets the fleet scale to zero. Max GPUs is the ceiling and cannot exceed your plan's GPUs-per-deployment limit; Min cannot be higher than Max. Both default to 1. Schedule windows can narrow these bounds.

Cold starts. When a deployment has no workers yet (a fresh start, a schedule reopen, or a restart while its first workers are still provisioning), CUDAnetes rents up to three candidates at once — never more than Max GPUs — and keeps the first one that becomes healthy. The others are destroyed, or drained first if they already have requests. This means one slow host cannot hold up the start. If several become healthy at once, the cheapest needed ones are kept. After that, scaling rents only the missing workers, and a paused fleet that is resumed is not hedged. Vast.ai bills the extra candidates for as long as they exist.

Machine spread (on by default) rents each worker on a different physical host, so one host failure takes down at most one worker. CUDAnetes excludes hosts already used by the fleet when looking for offers. If no distinct-host offer appears within machineSpreadGraceMinutes (15 by default), it starts allowing co-location to reach Min GPUs and emits a spread-exhausted event; strict spread resumes on the next rental. Turn it off if capacity is tight and you would rather share a host than wait.

Scale from zero. With Min GPUs 0, a requests fleet can have no workers. The first request after that gets a 503, because there is nothing to route it to. CUDAnetes counts that rejection as demand and starts provisioning a worker; requests succeed once it is ready.

Scheduling

Deployments run all the time by default. Add schedule windows to run the fleet only on chosen days and hours — useful for cost control when the workload doesn't need to be up around the clock.

Each window sets the days, a start and end time, and optional Min/Max GPU bounds. Overlapping windows combine: the highest Min and the lowest Max win. If the end is earlier than the start the window crosses midnight: Monday 22:00–06:00 runs Monday 22:00–24:00 and Tuesday 00:00–06:00 — the selected days are the days a window starts. Set days and times in your local time; CUDAnetes stores and evaluates schedules in UTC.

Outside every window the fleet is destroyed — no workers run and nothing is billed. The deployment stays running with zero workers; zero workers is a normal running state, so it provisions again automatically when a window reopens.

Security

One optional layer protects the proxy URL, not the instance itself:

proxyAuth

Requires Authorization: Bearer <authKey> on every request and answers 401 otherwise. The proxy consumes that header and does not forward it, so the container never receives it — use a different header for your app's own auth.

The public endpoint CUDAnetes gives you is the proxy URL; deployments are never served on the CUDAnetes app origin. Vast.ai also exposes each worker's direct http://… URL, shown on the deployment page and available in the Vast.ai instances console. That address uses plain HTTP and bypasses proxyAuth, hardCap and the account proxy quota. CUDAnetes does not expose SSH or create tunnels for you.

Secrets (vastApiKey, authKey, imageLogin) are stored encrypted at rest (AES-256-GCM), returned by the API only as has* flags, and never logged.

Proxy & URL

Every deployment is reachable at one stable URL:

https://proxy.cudanetes.com/<id>/

The path after <id>/ and the query string are preserved. A request to https://proxy.cudanetes.com/<id>/v1/chat?stream=true reaches the worker as http://<worker>/v1/chat?stream=true. The method and body are forwarded too; request bodies over 100 MB get 413.

Each request goes to one ready worker — the one with the fewest in-flight requests. Workers at their request limit and workers that are not ready are skipped. If no worker is available, the proxy returns 503.

The proxy does not retry a request on another worker. Once a worker is chosen, a connection failure returns 502, and a response from the worker is passed through as-is.

Limits

Plan limits

All features are included on every plan. Plans differ only in how many deployments you can have, how many Max GPUs each deployment can use, and deployment traffic (see Pricing). Deployments × Max GPUs sets the account GPU ceiling.

Deployment actions

Starting or stopping GPUs takes minutes, so deployment actions are rate-limited per account to prevent retry storms and overlapping lifecycle calls. All API keys share the bucket. Create 20/min, update 20/min, scale 20/min, delete 20/min. Start, stop, and pause 10/min each. Restart 6/min. A 429 includes Retry-After.

Deployment traffic

Each plan caps how many requests per minute can reach your deployments' public URLs. This is separate from the CUDAnetes API (see Pricing). By default there is no per-IP cap; set a positive proxyPerIpLimit in deployment settings to limit one client IP. The account limit always applies.

Cost optimization

CUDAnetes scans each deployment every scanIntervalMinutes (default 30) for a cheaper offer among the GPU types you allow. It compares the most expensive ready worker with offers that pass your placement and reliability filters. The optimizer can switch between those GPU types, so list only the cards you actually want. It replaces that worker only when the offer is at least minSavingsPercent cheaper — it never pays more than the worker it replaces.

The replacement is provisioned first. The old worker keeps serving until the new one is ready and stays healthy for settleMinutes (default 10), then it is drained. If the replacement does not become ready, the old worker is kept. Enable it under Cost & performance and set minSavingsPercent to avoid churn on marginal gains.

API reference

The API is served from the same origin as the app in production (https://cudanetes.com/v1/...). API keys (Authorization: Bearer <key>, created in the dashboard) authenticate the deployment endpoints below.

curl -H "Authorization: Bearer YOUR_API_KEY" https://cudanetes.com/v1/deployments

Deployments

Create a deployment

Required: name, dockerImage, gpuTypes (SKU tokens like RTX_4090:24) and vastApiKey. Everything else is optional. The GPU catalog (GET /v1/gpu) lists valid SKU tokens.

POST /v1/deployments
Content-Type: application/json

{
  "name": "my-nginx",
  "dockerImage": "nginx:latest",
  "gpuTypes": ["RTX_4090:24"],
  "vastApiKey": "your-vast-key",
  "containerPort": 80,
  "healthEndpoint": "/",
  "minInstances": 1,
  "maxInstances": 3,
  "maxPricePerHour": 0.5
}

The example uses nginx:latest: it listens on port 80 and serves /. For your own image, set containerPort and healthEndpoint to the port and path it actually serves.

Common optional fields and their defaults:

Placement and reliability filters such as minReliability, minCudaVersion, and geolocation map to Vast.ai offer-search fields — see Vast.ai search offers.

Errors

Errors are JSON. Status codes: 400 invalid input, 401 missing or invalid auth, 403 forbidden, 404 not found, 409 conflict (editing a non-stopped deployment, or an action not allowed in the current state), 429 rate limited (Retry-After tells you when to retry).

Deployment traffic uses the proxy URL, not this API host.

FAQ

Can I use private Docker images?

Yes. Set imageLogin to -u <user> -p <token> <registry>; it is stored encrypted and passed to Vast.ai as docker login arguments.

What happens if a GPU instance fails?

CUDAnetes detects unhealthy instances and automatically provisions replacements. Your container restarts on a fresh GPU.

How does auto-scaling work?

Set minInstances and maxInstances, then choose a policy. With scalingPolicy: "requests", CUDAnetes sizes the fleet from average concurrent requests and recent 503s over rolling windows, within those bounds. With the default manual policy, set desiredInstances yourself.

Why is my Vast.ai bill higher than the GPU price?

Vast bills GPU time, disk storage and network separately. The GPU $/hr does not include bandwidth: image and model pulls (download) and inference responses (upload) are billed per byte. Check the Vast.ai invoice breakdown and set a bandwidth cap.

Is Vast.ai reliable enough for production?

Hosts vary, but you do not need a perfect host. CUDAnetes health-checks and replaces workers, blacklists failing machines and can spread the fleet; several cheap workers are often more resilient than one “reliable” host. Verified, datacenter and reliability filters are optional — they shrink the pool. CUDAnetes does not provide an uptime SLA.

How do I avoid expensive bandwidth hosts?

Keep maxBandwidthCostPerTb at the default $10 or lower. The cap applies to both directions and prevents renting hosts with extreme rates, but it is a rate cap, not a monthly budget. The chosen host's rates appear in the provisioning event.

What bandwidth cap should I use?

$10/TB keeps most of the marketplace available and avoids the worst traps. $5/TB is stricter; $1/TB removes A100/H100 and most alternatives. No limit can expose you to rates above $50/TB.

Can I use spot/preemptible instances?

CUDAnetes currently rents on-demand offers only; Vast.ai spot/preemptible offers are not selectable. Lower maxPricePerHour for cheaper offers and raise minReliability for more stable hosts.

Do I need my own Vast.ai account?

Yes. CUDAnetes rents on your Vast.ai account with your API key; you keep Vast billing and CUDAnetes takes no GPU markup. The key is stored encrypted per deployment.

What happens when a Vast.ai host fails?

CUDAnetes probes each worker's health endpoint every 5 seconds. A worker that stops returning 2xx is marked unhealthy and replaced, and the failing host is blacklisted. Requests already in flight on that worker can fail — the proxy does not retry them across workers — but new requests go to healthy workers.

Can I use CUDAnetes with RunPod or another GPU provider?

Not today. CUDAnetes orchestrates Vast.ai only; RunPod users would need a Vast.ai account and API key first.

Troubleshooting

The URL returns 503

No ready worker yet — the first GPU may still be provisioning or loading, or the deployment is outside its schedule window. Check the worker rows and the events timeline; a fresh worker can take a few minutes.

The container never reaches ready

Usually the container isn't serving on containerPort, the healthEndpoint doesn't return a 2xx, or the image isn't linux/amd64. CUDAnetes blacklists the failing host and retries on a fresh one — fix the image and it settles.

Requests return 401

proxyAuth is on. Send Authorization: Bearer <authKey>.

Requests return 409

The deployment isn't accepting traffic (paused, pausing, stopping, or stopped). Start it when stopped or paused; wait for a transition to settle.

The Vast.ai key is rejected

Check the Vast.ai account balance and the key's permissions — it needs Miscellaneous and Instances Read/Write.

running but nothing is provisioned

Outside the schedule window the fleet is destroyed until the window reopens (see Schedule). It can also be at Max GPUs 0 or waiting for matching capacity.

Where are the container logs?

CUDAnetes does not stream container stdout. The dashboard shows the deployment's lifecycle events; use the Vast.ai instances console for instance logs. CUDAnetes's raw log viewer is operator-only.

Still stuck? Email [email protected] with your deployment ID and what you see — we'll do our best to help as soon as possible.