Docker Execution Environment
Vast.ai provides Linux docker instances. Your template (or CLI command) controls most of the parameters to the underlying docker create call, but most resource constraint options naturally must be configured automatically by our system.
We automatically assign CPU, RAM, shared mem, and related cgroup resources automatically in proportion to your instance's cost vs the total machine cost.
These launch modes attempt to 'inject' Jupyter/SSH setup in to your existing docker image. The image entrypoint is replaced, so if you are using Jupyter or SSH launch mode for a docker image that uses an entrypoint script, you typically want to take the entrypoint command and add it to the end of your onstart. Sometimes the injection scripts can cause obscure loading errors.If you run into such issues with a custom image it may be best to use the simpler args/entrypoint launch mode and setup ssh/jupyter yourself.
You can currently set 3 types of docker create/run options in the GUI and CLI:
environment variables: "-e JUPYTER_DIR=/ -e TEST=OK"
hostname: "-h billybob"
ports: "-p 8081:8081 -p 8082:8082/udp -p 70000:70000"
Use the -e docker syntax in the docker create/run options to set env variables. For example, to set the env variables TZC to UTC and TASKID to "TEST":
Any environment variables you set will be visible only to your onstart script (or your entrypoint for entrypoint launch mode). When using the SSH or Jupyter launch modes, your env variables will not be visible inside your SSH/tmux/Jupyter session by default. To make custom environment variables visible to the shell, you need to export them to /etc/environment.
Add something like the following to the end of your onstart to export any env variables containing an underscore '_':
Or to export all env variables:
Some special environment variables are used to signal to the interface:
OPEN_BUTTON_PORT: Set this to map the open button on the instance panel to a specific (external) port corresponding to the specified internal port.
For example:
Will map the open button to whatever external port maps to internal port 7860.
Our system also predefines some environment variables you can use:
CONTAINER_ID: The unique ID of your instance.VAST_CONTAINERLABEL: Also the unique name/ID of your instance.PUBLIC_IPADDR: The instance's public IP address.SSH_PUBLIC_KEY: Your SSH public key from the account page.CONTAINER_API_KEY: Per instance API key you can use to access some CLI commands from within the instance.GPU_COUNT: Number of GPU devices.
PYTORCH_VERSION: The pytorch version (if applicable)JUPYTER_TOKEN: The Jupyter access token.JUPYTER_SERVER_ROOT: The root directory for Jupyter (can't navigate above this!)JUPYTER_SERVER_URL: Configured jupyter server URL (usually https://0.0.0.0:8080/)
Port env variables:
VAST_TCP_PORT_22: The external public TCP port that maps to internal port 22 (ssh).VAST_TCP_PORT_8080: The external public TCP port that maps to internal port 22 (ssh).
For each internal TCP port request:VAST_TCP_PORT_X: The external public TCP port that maps to internal port X.
For each internal UDP port request:VAST_TCP_PORT_X: The external public UDP port that maps to internal port X.
You can also use ports 70000 and above for identity port mappings (see networking below).
Selecting the ssh launch mode will open and use port 22 internal by default, whereas jupyter will open and use port 8080 (in addition to 22 for ssh). There are several ways to open additional application ports:
Any EXPOSE commands in your docker image will be automatically mapped to port requests. You can also open custom ports for any docker image more dynamically using -p arguments in the docker create/run options box in the image config editor pop-up menu. To open ports 8081 (tcp) and 8082 udp, use something like this:
This will result in additional arguments to docker create/run to expose those internal ports, which will be mapped to random external ports. Any ports exposed in these docker options are in addition to ports exposed through EXPOSE commands in the docker image, and the ports 22 or 8080 which may be opened automatically for SSH or Jupyter.
After the instance has loaded, you can find the corresponding external public IP:port by opening the IP Port Info pop-up (button on top of the instance) and then looking for something like:
In this case, the public IP:port 65.130.162.74:33526 can be used to access anything you run on port 8081 inside the instance. As a simple test case, you can run a simple minimal web browser inside the instance with the following command:
Which you would then access in this example by loading 65.130.162.74:33526 in your web browser.
In some cases you may need an identity port map like 32001:32001 where external:internal are the same.
For this just use an out-of-range port above 70000:
These out of range requests will map to random external ports and matching internal ports. You can then find the resulting mapped port with the appropriate env variables like : $VAST_TCP_PORT_70000
