Skip to main content
Worker states represent the current operational status of your serverless worker instances. Understanding these states helps you monitor your workers’ health and manage your costs effectively.

Worker States in the Vast UI

The following table describes all available worker states that are shown to users in the UI: For more information on the specific breakdown of charges, please refer to Serverless pricing

Worker States in the Vast CLI and logs

The worker logs, accessible through the Vast CLI and UI, have more detailed descriptions of worker states to aid in debugging issues. These are described in the table below. The value in parentheses is the status string written to the logs and returned by the worker status APIs (REST and WebSocket). Each state change is written to the endpoint and workergroup logs as Worker <id>: <old state> -> <new state> (<reason>).

Automatic Recovery

The serverless engine handles unhealthy workers without user action:
  • Error: Workers in the Error state are either restarted or destroyed and replaced. A Ready worker whose reliability (the share of routed requests it completes) falls below 0.5 is moved to Error.
  • Timeouts: A worker that stays in a transitional state for too long is moved to Error. For example, a worker that is still Creating 5 minutes after its creation request succeeded, or still Stopping after 2 minutes.
  • Unavailable: If the machine frees up, the worker returns to Inactive. When a start attempt fails because the GPUs are taken, the engine retries every 90 seconds. After 10 failed attempts the worker is destroyed, or sooner if the endpoint is at max_workers and has no other inactive workers to start. While a worker is Unavailable, the engine may recruit other workers to cover its capacity.
  • Offline: If the machine reconnects, the worker resumes. A worker that stays Offline for 5 minutes is destroyed.
Unavailable and Offline workers are among the first workers removed when the endpoint scales down.