We currently support 3 launch modes: entrypoint, ssh, and jupyter.

For this launch mode we simply run a docker container from your specified image as is.
The docker image entrypoint is the main run process, which you can override in the GUI or CLI (--entrypoint).
You can also pass arguments to your entrypoint in the GUI or CLI (--args).
Entrypoint launch mode is suitable for GPU worker instances which receive tasks from your webserver.
As ssh/jupyter access is not provided, your docker image is responsible for setting up any such connections as needed.

For this launch option we setup an ssh connection using proxy and or direct connections where appropriate (mapping to port 22 internal).
If the machine supports open ports our system will try to setup both a direct ssh connection and a backup proxy connection.
This is mostly automatic under the hood, but it does require that your docker image is compatible with typical ssh daemon setup.

With the SSH launch option your docker image entrypoint is not called as we must override it.
Instead we allow you to specify an onstart script which is called as part of the new entrypoint.

So if you are using making a template from an existing docker image, you typically will want to find its entrypoint command and move that to the onstart.

For this launch option we setup a jupyter using a proxy and or direct connections where appropriate (mapping to port 8080 internal).
If the machine supports open ports our system will try to setup a direct jupyter connection with a custom HTTPS certificate.
This is mostly automatic under the hood, but it does require that your docker image is compatible with typical jupyter setup.

With the Jupyter launch option your docker image entrypoint is not called as we must override it.
Instead we allow you to specify an onstart script which is called as part of the new entrypoint.

Virtual Machines

Launch Modes

A lower level organization that lives within an Endpoint. It consists of a template (with extra filters for search), a set of GPU instances (workers) created from that template, and hyperparameters.

Worker_Groups

The highest level clustering of instances for the autoscaler, consisting of a named endpoint string, a collection of Worker groups, and hyperparameters.

Endpoints

The Vast PyWorker is a Python web server designed to run alongside a machine learning model instance, providing autoscaler compatibility.

PyWorker

Worker_Group

Endpoint

The minimum number of workers you want to keep "cold" (meaning stopped and fully loaded) when your group has no load.