Docker Execution Environment

vast ai provides linux docker instances your template (or cli command) controls most of the parameters to the underlying docker create call, but most resource constraint options naturally must be configured automatically by our system cpu, memory, shared mem we automatically assign cpu, ram, shared mem, and related cgroup resources automatically in proportion to your instance's cost vs the total machine cost ssh/jupyter launch modes these launch modes attempt to 'inject' jupyter/ssh setup in to your existing docker image the image entrypoint is replaced, so if you are using jupyter or ssh launch mode for a docker image that uses an entrypoint script, you typically want to take the entrypoint command and add it to the end of your onstart sometimes the injection scripts can cause obscure loading errors if you run into such issues with a custom image it may be best to use the simpler args/entrypoint launch mode and setup ssh/jupyter yourself docker create options you can currently set 3 types of docker create/run options in the gui and cli environment variables " e jupyter dir=/ e test=ok" hostname " h billybob" ports " p 8081 8081 p 8082 8082/udp p 70000 70000" environment variables use the e docker syntax in the docker create/run options to set env variables for example, to set the env variables tzc to utc and taskid to "test" e tz=utc e taskid="test" any environment variables you set will be visible only to your onstart script (or your entrypoint for entrypoint launch mode) when using the ssh or jupyter launch modes, your env variables will not be visible inside your ssh/tmux/jupyter session by default to make custom environment variables visible to the shell, you need to export them to /etc/environment add something like the following to the end of your onstart to export any env variables containing an underscore ' ' env | grep >> /etc/environment; or to export all env variables env >> /etc/environment; user account environment variables users can set their environment variables in their user account settings, which will automatically inject these variables into every container they use special env vars some special environment variables are used to signal to the interface open button port set this to map the open button on the instance panel to a specific (external) port corresponding to the specified internal port for example e open button port=7860 will map the open button to whatever external port maps to internal port 7860 jupyter port use this to control the jupyter button set this to your internal jupyter port and the ui will map the jupyter button to open jupyter on the corresponding ip in a new tab for example e jupyter port=8081 will map the jupyter button to whatever external port maps to internal port 8081 jupyter token use this to control the jupyter button set this to your jupyter token and the ui will map the jupyter button to open jupyter using the corresponding jupyter token in a new tab for example e jupyter token=token will use token as a value of your jupyter token data directory this env variable is used as the default src or dst directory for data copy operations predefined env vars our system also predefines some environment variables you can use container api key per instance api key you can use to access some cli commands from within the instance container id the unique id of your instance data directory location on the instance to copy data to/from gpu count number of gpu devices public ipaddr the instance's public ip address ssh public key your ssh public key from the account page pytorch version the pytorch version (if applicable) jupyter token the jupyter access token jupyter server root the root directory for jupyter (can't navigate above this!) jupyter server url configured jupyter server url (usually https //0 0 0 0 8080/) vast containerlabel also the unique name/id of your instance port env variables vast tcp port 22 the external public tcp port that maps to internal port 22 (ssh) vast tcp port 8080 the external public tcp port that maps to internal port 22 (ssh) for each internal tcp port request vast tcp port x the external public tcp port that maps to internal port x for each internal udp port request vast tcp port x the external public udp port that maps to internal port x you can also use ports 70000 and above for identity port mappings (see networking below) networking vast ai docker instances have full internet access, but generally do not have unique ip addresses instances can have public open ports, but as ip addresses are shared across machines/instances the public external ports are partitioned somewhat randomly in essence each docker instance gets a fraction of a public ip address based on a subset of ports each open internal port (such as 22 or 8080 etc) is mapped to a random external port on the machine's (usually shared) public ip address selecting the ssh launch mode will open and use port 22 internal by default, whereas jupyter will open and use port 8080 (in addition to 22 for ssh) there are several ways to open additional application ports custom ports note there is currently a limit of 64 total open ports per container/instance any expose commands in your docker image will be automatically mapped to port requests you can also open custom ports for any docker image more dynamically using p arguments in the docker create/run options box in the image config editor pop up menu to open ports 8081 (tcp) and 8082 udp, use something like this p 8081 8081 p 8082 8082/udp this will result in additional arguments to docker create/run to expose those internal ports, which will be mapped to random external ports any ports exposed in these docker options are in addition to ports exposed through expose commands in the docker image, and the ports 22 or 8080 which may be opened automatically for ssh or jupyter after the instance has loaded, you can find the corresponding external public ip by opening the ip port info pop up (button on top of the instance) and then looking for the external port which maps to your internal port it will have a format of public ip > internal port for example 65 130 162 74 33526 > 8081/tcp in this case, the public ip 65 130 162 74 33526 can be used to access anything you run on port 8081 inside the instance we strongly recommend you test your port mapping testing ports you can quickly test your port mapping with a simple command to start a minimal web server inside the instance with the following command python m http server 8081 which you would then access in this example by loading 65 130 162 74 33526 in your web browser this should open a file directory identity ports in some cases you may need an identity port map like 32001 32001 where external are the same for this just use an out of range port above 70000 p 70000 70000 p 70001 70001 these out of range requests will map to random external ports and matching internal ports you can then find the resulting mapped port with the appropriate env variables like $vast tcp port 70000 using the cli​ from inside a special instance api key should already be installed in your container and mapped to the env variable container api key the vastai cli may already be installed, but if not you can easily install with pip root\@c 38250 $ pip install vastai; then test it by starting the instance (which is a no op as the instance is already running) vastai start instance $container id; the instance apikey should already set, but if not, may need to specify it from env variable vastai start instance $container id api key $container api key; if that works then you can stop the instance as well vastai stop instance $container id; vastai stop instance $container id api key $container api key; you can also use destroy instance and a few other commands using the instance api key if $container id and or $container api key is not defined check your environment variables using the 'env' command if you are missing the predefined env variables from an ssh session you may need to add a command to export them to /etc/environment (see earlier section on env variables) if you don't have the instance api key for whatever reason, you can also generate it first run the following from inside the instance to create a special per instance api key and save it in the appropriate location cat / ssh/authorized keys | md5sum | awk '{print $1}' > ssh key hv; echo n $vast containerlabel | md5sum | awk '{print $1}' > instance id hv; head c 1 q ssh key hv instance id hv > / vast api key; then you should be able to run start/stop without passing in the key vastai start instance $container id;