This guide will help you create custom templates for running containers on Vast.ai. There are three approaches:

Vast instances are currently Linux Docker containers and Ubuntu virtual machines. Many Linux based images should work within Vast.

This approach lets you customize recommended templates without building a custom image.

Example PROVISIONING_SCRIPT:

This script will run on first boot to set up your environment. All installations should go to /workspace/ for proper persistence.

The base-image template includes PORTAL_CONFIG for secure application access management. This environment variable controls how applications are exposed and accessed.

The structure of this variable is:

Example:

The hostname in Docker instances will always be localhost

Where the internal port and local port are not equal then Caddy will be configured to listen on 0.0.0.0:external_port acting as a reverse proxy for hostname:local_port

If the external_port and local_port are equal then Caddy will not act as a proxy but the Instance Portal UI will still create links. This is useful because it allows us to create links to Jupyter which is not controlled by Supervisor in Jupyter Launch mode.

url_path will be appended to the instance address and is generally set to / but can be used to create application deep links.

The caddy_manager script will write an equivalent config file at /etc/portal.yaml on boot if it does not already exist. This file can be edited in a running instance.

Important: When defining multiple links to a single application, only the first should have non equal ports - We cannot proxy one application multiple times.

Note: Instance Portal UI is not required and its own config declaration can be removed from PORTAL_CONFIG. This will not affect the authentication system.

You can use existing Docker Hub images, but please note:

Once logged in, click on Templates to go to the template section. This is where you can create, manage, and edit templates.

Click the button + New to create a new template.

In the Image Path Tag, you can add the path/tag of the docker image you wish to use. The Version Tag is where you would set the version.

You can use the -e option to set environment variables in the Docker Options field by adding something like:

-e MYVAR1

You can also add environment variables one by one in the Environment Variables section. These entries will automatically sync with the Docker Options field. Make sure to click "+" adding the environment variable and value.

These options will be appended to the docker run command before the docker run command is executed within one of the machines. Similar to how it would be if you were executing "docker run -e MYVAR1 IMAGE[|@DIGEST]" on your local machine.

Docker Run Env Option Documentation

In addition to setting environment variables, you can also open custom ports by adding -p ${HOST_MACHINE_PORT}:${CONTAINER_PORT} to the Docker Options section.

You can also add them to the Ports section. Once more, this will automatically sync with the Docker Options field.

FAQ: How can I Open Custom Ports

You can see what's happening inside a container at particular port by finding the public ip address mapped to that container port.

How to View Instance's IP Addresses and Open Ports

More details on How to View Instance's IP Addresses and Open Ports

Internally, one of the machine's open ports is mapped to this container port.

You can use ssh to open a ssh connection to the host machine's public ip address. The machine will forward traffic from the host machine's public port to the container port you specified.

You can use a ssh command like the one below.

You can choose the jupyter, ssh, or entrypoint launch mode depending on your needs.

Vast Launch Modes

These commands will be executed as soon the docker container starts, similar to how a bash script be would be executed suppose you created a bash script and added a CMD command in the Dockerfile so Docker would execute this bash script when the container starts.

In this case: CMD specifies the default command to run when the container starts, which is your start.sh script.

You can also try to overwrite existing files built into the image. Make sure you can switch to a user that has write permissions to that particular file.

For example, you can remove all instances of '-sslOnly' in a particular file using sed.

You can also make directories.

Make sure to append environment variables to /etc/environment file in your on-start section because this makes environment variables available to all users and processes and ensures they are persisted even if your instance/docker container is rebooted. This code snippet will take care of this.

Also make sure to find the image’s ENTRYPOINT or CMD command and call that command at the end of the on-start section. We overwrite that command to set up jupyter/ssh server, etc. under the hood.

You don't have to specify docker.io as the server name if your repository is Docker Hub. Docker automatically uses docker.io to pull the image if no other registry is specified.

You do have to specify your server name if your repository is something else.

For example,

GitHub Container Registry (GHCR) - Server Name: ghcr.io

Google Container Registry (GCR) - Server Name: gcr.io

We will have the same character set for docker login usernames and passwords as Docker soon.

Choose whether to include ReadMe, make template public, and fill out readme if you desire.

A new template will be created when you edit, make changes, and save.

You can download and upload files: Copy Data To/From Instance Storage

You can also backup data to cloud storage: Backup Data to Cloud Storage

Creating Templates for GROBID