​
How To: Create Your Own Template

​
Introduction

This guide will help you create custom templates for running containers on Vast.ai. There are three approaches:
  1. Using PROVISIONING_SCRIPT with recommended templates (simplest)
  2. Create Dockerfile from Vast.ai base images (recommended for custom images)
  3. Using existing Docker images (with security considerations)
Vast instances are currently Linux Docker containers and Ubuntu virtual machines. Many Linux based images should work within Vast.

​
Prerequisites

​
Approach 1: Using PROVISIONING_SCRIPT (Simplest)

This approach lets you customize recommended templates without building a custom image.

​
Steps

  1. Go to the Templates tab in Vast.ai interface
  2. Search for “base-image” or “Pytorch” depending on your needs:
    • vast-ai/base-image is a general purpose image
    • vast-ai/pytorch is a base image for working with PyTorch-based applications on Vast
  3. Click “Edit” on your chosen template
  4. Add the PROVISIONING_SCRIPT environment variable:
    • In the Environment Variables section, add a new variable named “PROVISIONING_SCRIPT”
    • The value should be a URL pointing to a shell script (from GitHub, Gist, etc.)
      • If you’re specifying a shell script stored in GitHub repo, specify url like
Url
https://raw.githubusercontent.com/karthik-vast-ai/vast-cli/distributed-inference-integration/provisioning_script.sh7
  1. Make sure to click ”+” to add the environment variable
  2. Click Create and Use

Add PROVISIONING_SCRIPT variable

Example PROVISIONING_SCRIPT:
Bash
#!/bin/bash

# 
cd /workspace/
# Cause the script to exit on failure.
set -eo pipefail

# Activate the main virtual environment
. /venv/main/bin/activate

# Install your packages
pip install your-packages

# Download some useful files
wget -P "${WORKSPACE}/" https://example.org/my-application.tar.gz
tar xvf ${WORKSPACE}/my-application.tar.gz"

# Set up any additional services
echo "my-supervisor-config" > /etc/supervisor/conf.d/my-application.conf
echo "my-supervisor-wrapper" > /opt/supervisor-scripts/my-application.sh
chmod +x /opt/supervisor-scripts/my-application.sh

# Reconfigure the instance portal
rm -f /etc/portal.yaml
export PORTAL_CONFIG="localhost:1111:11111:/:Instance Portal|localhost:1234:11234:/:My Application"

# Reload Supervisor
supervisorctl reload
This script will run on first boot to set up your environment. All installations should go to /workspace/ for proper persistence.

​
Configuring Application Access with PORTAL_CONFIG

The base-image template includes PORTAL_CONFIG for secure application access management. This environment variable controls how applications are exposed and accessed.
PORTAL_CONFIG structure
hostname:external_port:local_port:url_path:Application Name|hostname:external_port:local_port:url_path:Application Name
The structure of this variable is:
  • Each application is separated by the | character
  • Each application parameter is separated by the : character
  • Each application must specify hostname:external_port:local_port:url_path:Application Name
Example:
Bash
"localhost:8002:18002:/hello:MyApp|localhost:1111:11111:/:Instance Portal|localhost:8080:18080:/:Jupyter|localhost:8080:8080:/terminals/1:Jupyter Terminal|localhost:8384:18384:/:Syncthing|localhost:6006:16006:/:Tensorboard"
The hostname in Docker instances will always be localhost Where the internal port and local port are not equal then Caddy will be configured to listen on 0.0.0.0:external_port acting as a reverse proxy for hostname:local_port If the external_port and local_port are equal then Caddy will not act as a proxy but the Instance Portal UI will still create links. This is useful because it allows us to create links to Jupyter which is not controlled by Supervisor in Jupyter Launch mode. url_path will be appended to the instance address and is generally set to / but can be used to create application deep links. The caddy_manager script will write an equivalent config file at /etc/portal.yaml on boot if it does not already exist. This file can be edited in a running instance. Important: When defining multiple links to a single application, only the first should have non equal ports - We cannot proxy one application multiple times. Note: Instance Portal UI is not required and its own config declaration can be removed from PORTAL_CONFIG. This will not affect the authentication system.

​
Approach 2: Create Dockerfile from Vast.ai Base Images

​
Why Use Vast.ai Base Images?

  • Built-in security features through Caddy proxy
  • Automatic TLS encryption for web services
  • Authentication token protection
  • Proper isolation between external and internal services

​
Steps

  1. Start with a Vast.ai base image or Vast.ai Pytorch base image in your Dockerfile:
Dockerfile
#For example 
FROM vastai/base-image:cuda-12.6.3-cudnn-devel-ubuntu22.04-py313
# or
FROM vastai/pytorch:2.6.0-cuda-12.6.3-py312

# Install your applications into /opt/workspace-internal/
# This ensures files can be properly synced between instances
WORKDIR /opt/workspace-internal/

# Activate virtual environment from base image 
RUN . /venv/main/bin/activate

RUN your-installation-commands
  • Build and push your image to a container registry
  • Create a new template using your custom image
    • See next approach for steps on how to do that

​
Approach 3: Create Template Using Existing Docker Images

You can use existing Docker Hub images, but please note:
  • Running external-facing services without proper security measures is not recommended
  • Consider wrapping the service in Vast.ai’s security infrastructure

​
Step 1 - Navigate to the ‘Templates’ Section

Once logged in, click on Templates to go to the template section. This is where you can create, manage, and edit templates.

​
Step 2 - Create a New Template

Click the button + New to create a new template.
Create

Create button

​
Step 3 - Specify the image path/tag of your docker image

In the Image Path Tag, you can add the path/tag of the docker image you wish to use. The Version Tag is where you would set the version.
Image&Tag

Image Tag

​
Step 4 - Set Environment Variables

You can use the -e option to set environment variables in the Docker Options field by adding something like: -e MYVAR1 You can also add environment variables one by one in the Environment Variables section. These entries will automatically sync with the Docker Options field. Make sure to click ”+” adding the environment variable and value.
Env&Stuff

Env\&Stuff

These options will be appended to the docker run command before the docker run command is executed within one of the machines. Similar to how it would be if you were executing “docker run -e MYVAR1 IMAGE[|@DIGEST]” on your local machine. Docker Run Env Option Documentation

​
Step 5 - [Optional] Open Custom Ports

In addition to setting environment variables, you can also open custom ports by adding -p \{HOST\_MACHINE\_PORT}:{CONTAINER_PORT} to the Docker Options section. You can also add them to the Ports section. Once more, this will automatically sync with the Docker Options field.
Env&Stuff

Env\&Stuff

FAQ: How can I Open Custom Ports You can see what’s happening inside a container at particular port by finding the public ip address mapped to that container port. How to View Instance’s IP Addresses and Open Ports More details on How to View Instance’s IP Addresses and Open Ports Internally, one of the machine’s open ports is mapped to this container port. You can use ssh to open a ssh connection to the host machine’s public ip address. The machine will forward traffic from the host machine’s public port to the container port you specified. You can use a ssh command like the one below.
Text
ssh -p [SSH_PORT] [USER]@[REMOTE_HOST] -L [LOCAL_PORT]:localhost:[REMOTE_PORT]
ssh -p 22 user@remote.example.com -L 8080:localhost:8080

​
Step 6 - Pick a Launch Mode

You can choose the jupyter, ssh, or entrypoint launch mode depending on your needs. Vast Launch Modes

​
Step 7 - Add Starting Bash Commands to On-start Script Section

These commands will be executed as soon the docker container starts, similar to how a bash script be would be executed suppose you created a bash script and added a CMD command in the Dockerfile so Docker would execute this bash script when the container starts.
  1. Before Vast
Text
# Use a base image
FROM ubuntu:20.04

# Make the script executable
RUN chmod +x /usr/local/bin/start.sh

# Set the script as the default command
CMD ["/usr/local/bin/start.sh"]
In this case: CMD specifies the default command to run when the container starts, which is your start.sh script.
  1. In On-Start Script Section With Vast:
Text
chmod +x /usr/local/bin/start.sh
bash /usr/local/bin/start.sh
You can also try to overwrite existing files built into the image. Make sure you can switch to a user that has write permissions to that particular file. For example, you can remove all instances of ‘-sslOnly’ in a particular file using sed.
Text
sed -i 's/-sslOnly//g' /dockerstartup/vnc_startup.sh
You can also make directories.
Text
sudo -i -u kasm-user mkdir -p /home/kasm-user/Desktop
Exampleonstart

Exampleonstart

Make sure to append environment variables to /etc/environment file in your on-start section because this makes environment variables available to all users and processes and ensures they are persisted even if your instance/docker container is rebooted. This code snippet will take care of this.
Text
env >> /etc/environment
Also make sure to find the image’s ENTRYPOINT or CMD command and call that command at the end of the on-start section. We overwrite that command to set up jupyter/ssh server, etc. under the hood.

​
Step 8 - Specify Docker Repository Server Name if Necessary

You don’t have to specify docker.io as the server name if your repository is Docker Hub. Docker automatically uses docker.io to pull the image if no other registry is specified. You do have to specify your server name if your repository is something else. For example, GitHub Container Registry (GHCR) - Server Name: ghcr.io Google Container Registry (GCR) - Server Name: gcr.io

​
Step 9 - Specify Docker Login Username and Password

We will have the same character set for docker login usernames and passwords as Docker soon.
Dockerlogin

Dockerlogin

​
Step 10 - Specify Template Name and Description

Templatename

Templatename

​
Step 11- Select the Amount of Disk Space You Want

Diskspace2

Diskspace2

​
Step 12 - Fill Out Accessibility Section

Choose whether to include ReadMe, make template public, and fill out readme if you desire.
Readme

Readme

​
Step 13 - Choose to Save And/Or Select Your Template

A new template will be created when you edit, make changes, and save.
Createbuttons

Createbuttons

​
[Optional] Download/Upload Files/Backup To Cloud Storage

You can download and upload files: Copy Data To/From Instance Storage You can also backup data to cloud storage: Backup Data to Cloud Storage

​
Troubleshooting

  • If your image is built for a different CPU architecture than your Vast machine, then it won’t work. You can try to find a machine with the required CPU architecture using our GUI or CLI. CLI Documentation Command to Search Offers Using CLI
  • If your image requires a higher CUDA version, then look for a machine with a higher Max CUDA version. The Max CUDA version can be found on the instance card. How to Find Max CUDA version in Instance Card
  • If your image is built to run jupyter, then try running it on a port different than 8080.
  • If you are having issues using ssh launch mode, add your public key to ~/.authorized_keys, install openssh, start openssh when the container starts, and forward the ssh server’s port.

​
Template Examples

Creating Templates for GROBID