CPU, Memory, Shared Mem
We automatically assign CPU, RAM, shared mem, and related cgroup resources automatically in proportion to your instance’s cost vs the total machine cost.SSH/Jupyter Launch Modes
These launch modes attempt to ‘inject’ Jupyter/SSH setup in to your existing docker image. The image entrypoint is replaced, so if you are using Jupyter or SSH launch mode for a docker image that uses an entrypoint script, you typically want to take the entrypoint command and add it to the end of your onstart. Sometimes the injection scripts can cause obscure loading errors. If you run into such issues with a custom image it may be best to use the simpler args/entrypoint launch mode and setup ssh/jupyter yourself.Docker Create Options
You can currently set 3 types of docker create/run options in the GUI and CLI: environment variables: “-e JUPYTER_DIR=/ -e TEST=OK” hostname: “-h billybob” ports: “-p 8081:8081 -p 8082:8082/udp -p 70000:70000”Environment Variables
Use the -e docker syntax in the docker create/run options to set env variables. For example, to set the env variables TZC to UTC and TASKID to “TEST”:Text
Text
Text
User Account Environment Variables
Users can set their environment variables in their User Account settings, which will automatically inject these variables into every container they use.Special Env Vars
Some special environment variables are used to signal to the interface: OPEN_BUTTON_PORT: Set this to map the open button on the instance panel to a specific (external) port corresponding to the specified internal port. For example:Text
Text
Text
Predefined Env Vars
Our system also predefines some environment variables you can use: CONTAINER_API_KEY: Per instance API key you can use to access some CLI commands from within the instance. CONTAINER_ID: The unique ID of your instance. DATA_DIRECTORY: Location on the instance to copy data to/from GPU_COUNT: Number of GPU devices. PUBLIC_IPADDR: The instance’s public IP address. SSH_PUBLIC_KEY: Your SSH public key from the account page. PYTORCH_VERSION: The pytorch version (if applicable) JUPYTER_TOKEN: The Jupyter access token. JUPYTER_SERVER_ROOT: The root directory for Jupyter (can’t navigate above this!) JUPYTER_SERVER_URL:Configured jupyter server URL (usually https://0.0.0.0:8080/)
VAST_CONTAINERLABEL: Also the unique name/ID of your instance.
Port env variables:
VAST_TCP_PORT_22: The external public TCP port that maps to internal port 22 (ssh).
VAST_TCP_PORT_8080: The external public TCP port that maps to internal port 22 (ssh).
For each internal TCP port request:
VAST_TCP_PORT_X: The external public TCP port that maps to internal port X.
For each internal UDP port request:
VAST_TCP_PORT_X: The external public UDP port that maps to internal port X.
You can also use ports 70000 and above for identity port mappings (see networking below).
Networking
Vast.ai docker instances have full internet access, but generally do not have unique IP addresses. Instances can have public open ports, but as IP addresses are shared across machines/instances the public external ports are partitioned somewhat randomly. In essence each docker instance gets a fraction of a public IP address based on a subset of ports. Each open internal port (such as 22 or 8080 etc) is mapped to a random external port on the machine’s (usually shared) public IP address. Selecting the ssh launch mode will open and use port 22 internal by default, whereas jupyter will open and use port 8080 (in addition to 22 for ssh). There are several ways to open additional application ports:Custom Ports
Note: there is currently a limit of 64 total open ports per container/instance. Any EXPOSE commands in your docker image will be automatically mapped to port requests. You can also open custom ports for any docker image more dynamically using -p arguments in the docker create/run options box in the image config editor pop-up menu. To open ports 8081 (tcp) and 8082 udp, use something like this:Text
Text
Testing Ports
You can quickly test your port mapping with a simple command to start a minimal web server inside the instance with the following command:Text
Identity Ports
In some cases you may need an identity port map like 32001:32001 where external are the same. For this just use an out-of-range port above 70000:Text
Using the CLI from Inside
A special instance api key should already be installed in your container and mapped to the env variable CONTAINER_API_KEY. The vastai CLI may already be installed, but if not you can easily install with pip:Text
Text
Text
Text
Text
Text