 code, which is configured to support several backends. The currently supported backends are:

Each backend has its own launch script and expected environment variables. We provide custom templates that configure everything needed to set up an autogroup for these backends, available 

. To modify default parameters, refer to the README files for each backend in the Vast PyWorker repository:

After launch, these images run inference code in a separate process accessible as an API server. The Vast PyWorker backend code wraps HTTP calls to this inference process, meaning each backend can support any endpoint that the underlying inference API server supports. However, each endpoint requires defining a wrapper function in the appropriate 

To add an endpoint to an existing backend, follow the instructions in the 

. This guide's architecture can also be used to write new backends, where the underlying inference code is accessible through an HTTP server in its own process.

You can see the currently supported endpoints for each backend:

To support additional endpoints, follow the guide and fork the repository to add your own code. These wrapper endpoints have the same name, input arguments, and output format as the underlying endpoint.

The authentication information returned by 

must be included in the request JSON to the endpoint but will be filtered out before forwarding to the backend server. When Vast’s autoscaling server returns a server address from the 

endpoint, it provides a unique signature with your request. The authentication server verifies this signature to ensure that only authorized clients can send requests to your server.

Backends