Templates

what is a template? a template is how vast helps you launch an instance, setting up your rented machine with whatever software and formatting you need templates are generally used for launching instances through the web interface, but they can also be used in the cli or through the api in this document, we will focus on the web interface, but we will link to other relevant documentation thoughout in the simplest technical terms, you can consider a template to be a wrapper around docker run the template contains all of the information you want to pass to our systems to configure the environment you can browse the template section of the web interface at cloud vast ai/templates recommended templates we provide several recommended templates to help you get started these are pre configured environments that you can use as is, or you can tweak them to your own requirements it's a great idea to look at how these templates have been configured to guide you in creating your own find out more about our recommended templates in the vast templates https //docs vast ai/templates#oh7ms section later in the guide quick start to jump right in and run a template, follow these steps visit the templates section of the console where you will find all of our recommended templates recommended templates page browse through the templates until you find one that meets your requirements in this guide we will use nvidia cuda, which is the first on the list it's a great starter template as it just includes the cuda development environment, plus a few extras to improve the user experience nvidia cuda template whos play button in bottom left corner now, click the 'play' button this will load the template and take you to the available offers offers page with gpu filter active there are filters available at the top of the page to help you target a particular gpu you will also find many additional filters on the left of the page for more fine grained control over the instances you find when you have found a suitable offer, simply click the 'rent' button to create your new instance you can now visit cloud vast ai/instances https //cloud vast ai/instances/ where you will find your running instance it may take a few minutes to be ready as everything is being set up instance view with blue open button when it is ready you will see the blue open button this indicates that the instance is ready to connect the action of the open button depends on the template you have chosen in this example you will be transferred to the instance portal later in this guide we will discuss how this can be configured creating your own template you can create a new template from scratch, or you can edit an existing template for this gude we will edit the nvidia cuda template we used in the quick start section from the templates page, click the pencil icon on the template card to open up the template editor you'll see two tabs config and readme we'll start with the config tab identification the first section helps you to keep your templates organized identification section of the template editor template name this will be displayed in bold on the template card choose something that helps you identify the template amongst your other templates for this guide, we will change the name to 'nvidia cuda demo' template description this field helps describe the function and purpose of the template completely optional for your own purposes, but very helpful if you intend to make this template public or share it with others docker repository and environment this is where you define the docker image you want to run, along with any options we want to pass into the container docker section of the template editor image path\ tag here is where you can define the docker image to run this field must be in the format repository/image name\ tag many of our templates pull from dockerhub but you can use any container registry just remember to add the full path if you're using an alternative registry eg nvcr io/nvidia/pytorch 25 04 py3 version tag for many registries we are able to pull the available list of tags so this field allows you to quickly select another version there is also a special \[automatic] tag you can use with this selected, the machine you choose for your instance will pull the most recent docker image that is compatible with that machine's own cuda version this will only work if the image tag contains the cuda version string for example my image cuda 12 8 would be loaded on a machine supporting cuda 12 8, but a machine with only cuda 12 6 would pull my image cuda 12 6 docker options this field is a textual representation of the ports and environment variables declared in the sections beneath it you can edit it directly or you can use the page widgets this field will only accept ports and environment variables other docker run options will be ignored ports to access your instance via the external ip address, you will need to add some ports to the template you can add both tcp and udp ports when your instance is created, a port will be randomly assigned to the external interface which will map into the instance port you selected environment variables here you can add any environment variables that your docker image requires do not save any sensitive information here if you are planning to make the template public place any variables with sensitive values into the environment variables section of your account settings page they will then be made available in any instance you create, regardless of the template used you can find out more about port mapping and special environment variables in our docker execution environment guide select launch mode templates offer three launch modes you can select from our recommened templates will usually launch in jupyter mode for easiest access, but you are free to choose whichever suits your needs launch mode selection options jupyter python notebook + ssh when you run the template in this mode, we will install jupyter and ssh at runtime jupyter will be available on mapped port 8080 and ssh will be available on mapped port 22 interactive shell server, ssh as above, but ssh only with no jupyter installation in both jupyter and ssh mode, the docker entrypoint for your image will not be run it will be replaced with our instance setup script so you should use the on start section (documented below) to start any services docker entrypoint in this mode, your docker image will run precisely as it is we will not include any additional software or access methods if your docker image does not offer ssh or another appropriate interface, please select one of the alternative modes if you need to interact with the running instance an additional field will be showin when using this launch mode to allow passing arguments to the image entrypoint field allowing for argument passing on start script here you can enter a short bash script which will be run during instance startup it is only available when using the jupyter or ssh launch modes, and is most useful for starting any services that your docker image would have launched if run if the entrypoint had been executed in our example 'nvidia cuda' template we are simply calling the entrypoint sh script which contains the logic for preparing the instance extra filters use this area to place restrictions on the machines that should show up in the search page when the template is selected extra filters showing this template is configured for both amd64 and arm64 cpus docker repository authentication if you are using a private docker image then you will need to add authentication credentials so the machine running the instance can download it disk space by setting the disk space in the template, you can ensure that new instances created from the template with use this amount as a minimum template visibility any template marked as public will be available in the template search system, while private images will not private templates can still be used by others if you have shared the template url never save a template as public if it contains sensitive information or secrets use the account level environmnet variables as an alternative cli command templates can be translated directly into cli launch commands this read only area shows what you would need to type or copy to the cli if you wanted to programatically launch an instance this way launch a template via the cli to learn more about starting instance from the cli, check out our quickstart guide save the template finally, you can save the template if you are creating a new template or editing one which is not associated with your account such as one of our recommended templates the buttons you see will be labelled 'create' for your own templates, you will see them labelled 'save' buttons for saving the 'create' button will create a copy of the templaye in the 'my templates' section of the templates page for you to use later the 'create & use' button will save the template, load it and then open up the offers page updating a template if you want to make changes to a template you previously saved, simply navigate back to the templates page and select 'my templates' here you'll be able to make your changes by clicking the pencil icon my templates showing the nvidia cuda demo template sharing a template it's really easy to share your template with other users we have two special links you can use and both include your referral code so you can earn if new users sign up find more about that here to share, click the three dots icon in the bottom right of the template card menu shows sharing options copy referral link this will copy a link that contains your referral id, creator id and the template name it will always point to the most recent template you created with this name really useful if you want people clicking the link to always get the most recent version copy template link this will copy a link containing your referral id and the template hash id it points to this specific template at this point in time templates all have a unique hash after every save this is useful as it allows you to find a previous version if you have tracked the hash id, but for sharing you probably want the referral link above remember to add a comprehensive readme to your template if you're going to share it this will help users to get started easily vast templates templates that we have created will often be found in the recommended section as they go through a qa process to ensure they function correctly and provide a positive user experience they contain extras not found in all templates so we will address the differences here instance portal you will notice that after clicking the open button on template running one of the vast ai templates a screen like this will appear instance portal landing page this is our instance portal and it provides easy to access links to services running in your instance it places an authentication layer in front of these services to prevent access by anyone who does not have the correct authentication token you are also able to create tunnels to your services without exposing ports full documentation for the instance portal is available here provisioning script vast ai templates support running a remote script on start to help configure the instance and download models and extensions that may not already be available in the docker image to use this feature, simply add environment variable provisioning script and set the value to a plain text shell script url the file will be downloaded and run on first start of the instance use the provisioning script to add a twist to an existing recommended template by downloading custom models and extensions base image you can find the source code for the docker images we used to create the vast templates on github these are large docker images that contain cuda development libraries, node + npm, opencl and other useful libraries despite their large size you'll find they generally start quickly because they have been cached on many of the host machines this makes them ideal for using as a base for your own images virtual machine templates currently we offer two vm templates; a cli and a desktop gui these differ from a standard template because they launch a full virtual machine environment rather than a docker container you can edit these templates as described above, but you should not change the docker image field only the images we distribue from docker io/vastai/kvm will work, but feel free to add extra environment variables and ports use the vm templates when you need to run applications that require namespace support run more than one docker container in an instance load kernel modules or run profiling jobs mount remote drives with rclone or similar advanced configuration for advanced template configuration options and variables, please see the docker execution environment guide