Introduction
What is Vast.ai?
Vast.ai is a cloud computing, matchmaking, and aggregation service focused on lowering the price of compute-intensive workloads. Our software allows anyone to easily become a host by renting out their hardware. Our web search interface allows users to quickly find the best deals for compute according to their specific requirements.What is the Secure Cloud (Only Trusted Datacenters) filter?
Vast.ai partners with vetted datacenter providers all over the globe. These partners have their equipment in certified locations that are current with ISO 27001 and/or Tier 3/4 standards. Vast.ai has verified that this equipment is in these facilities and that their certifications are up to date.For sensitive or production workloads, we recommend checking the “secure cloud” filter. Look for the blue datacenter label.How does Vast.ai work in a nutshell?
Hosts download and run our management software, list their machines, configure prices, and set any default jobs. Clients then find suitable machines using our flexible search interface, rent their desired machines, and finally run commands or start SSH sessions with a few clicks.What are Vast’s advantages?
Vast.ai provides a simple interface to rent powerful machines at the best possible prices, reducing GPU cloud computing costs by ~3x to 5x. Consumer computers and consumer GPUs, in particular, are considerably more cost-effective than equivalent enterprise hardware. We are helping the millions of underutilized consumer GPUs around the world enter the cloud computing market for the first time.What operating systems are provided? Windows?
Vast.ai provides a simple interface to rent powerful machines at the best possible prices, reducing GPU cloud computing costs by ~3x to 5x. Consumer computers and consumer GPUs, in particular, are considerably more cost-effective than equivalent enterprise hardware. We are helping the millions of underutilized consumer GPUs around the world enter the cloud computing market for the first time.What operating systems are provided? Windows?
Vast currently provides Linux Docker instances, mostly Ubuntu-based, no Windows.What interface is provided?
Currently, Vast has SSH access (for SSH instances), Jupyter instances with Jupyter GUI, or a command-only instance mode. We do not provide remote desktop.DLPerf
What is DLPerf?
DLPerf (Deep Learning Performance) - is our own scoring function. It is an approximate estimate of performance for typical deep learning tasks. Currently, DLPerf predicts performance well in terms of iters/second for a few common tasks such as training ResNet50 CNNs. For example, on these tasks, a V100 instance with a DLPerf score of 21 is roughly ~2x faster than a 1080Ti with a DLPerf of 10. It turns out that many tasks have similar performance characteristics, but naturally, if your task is very unusual in its compute requirements, the DLPerf score may not be very predictive. A single score can never be accurate for predicting performance across a wide variety of tasks; the best we can do is approximate performance on many tasks with a weighted combination. Although far from perfect, DLPerf is more useful for predicting performance than TFLops for most tasks.Rental Types
We currently offer two rental types: On Demand (High Priority) and Interruptible (Low Priority). On-Demand instances have a fixed price set by the host and run for as long as the client wants. Interruptible instances use a bidding system: clients set a bid price for their instance; the current highest bid is the instance that runs, the others are paused.Are vast.ai interruptible instances the same as AWS spot or GCE interruptible?
They are similar but have a few key differences. AWS spot instances and GCE interruptible instances both can be interrupted by on-demand instances, but they do not use a direct bidding system. In addition, GCE interruptible instances can only run for 24 hours. Vast.ai interruptible instances use a direct bidding system but are otherwise not limited.What happens when my interruptible instance loses the bid?
If another user places a higher bid or creates an on-demand rental for the same resources, then your instance will be stopped. Stopping an instance kills the running processes, so when you are using interruptible instances, it’s important to save your work to disk. Also, we highly recommend having your script periodically save your outputs to cloud storage as well because once your instance is interrupted, it could be a long wait until it resumes.Instances
How can I find OPEN_BUTTON_TOKEN in my container?
SSH into the machine where your contaner is running (instance). Or open Jupyter terminal via instance portal. Run the following command to print the token:How can I restart my programs once the instance restarts?
If you use the “custom command” option, then your command will run automatically when the instance starts up. However, if you are using an SSH instance, there is no default startup command. You can put startup commands in “/root/onstart.sh”. This startup script will be found and run automatically on container startup.I see my instance has a Lifetime - what does that mean?
Every instance offer on the Create page has a Max Duration. When you accept an offer and create an instance, this Max Duration becomes the instance lifetime and begins ticking down. When the lifetime expires, the instance is automatically stopped. The host can extend the contract, which will add more lifetime to your instance, or they may not - it’s up to them. Assume your instance will be lost once the lifetime expires; copy out any important data before then.How can I set environment variables?
Use the -e docker syntax in the docker create/run options. For example, to set the env variables TZC to UTC and TASKID to “TEST”:How can I stop the instance from within the instance?
A special instance api key should already be installed in your container. You can just install the vastai CLI and use the stop command:How can I launch another Docker container from within the instance?
Vast currently does not currently support Docker within Docker due to security constraints. You will need to launch each docker container on a separate instance.Networking
How can I open custom ports?
Add -p arguments in the docker create/run options box in the template configuration or image config editor pop-up menu. To open ports 8081 and 8082, use something like this:How can I open an identity port map like 32001:32001 where external are the same?
Just use an out-of-range port above 70000:Jupyter
I’m getting very slow transfer speeds using the jupyter download/upload?
You probably created a proxy instance, which is the default. The proxy instance is still useful for some use cases as you can get full speed downloading files with wget, git, external ftp, cloud storage, etc. However the built-in upload/download buttons can be very slow (especially when the proxy servers are overloaded). So if you want full speed transfers using the jupyter upload/download GUI, you need to create a direct https instance. On the create page, select EDIT IMAGE & CONFIG, open your image template (usually pytorch), then select the direct https mode under jupyter. Then create a new instance. You will need to import a certificate (see below).What is this HTTPS website unsecure warning?
The jupyter direct HTTPS option is faster than the proxy option, but it requires installing a new certificate in your browser: Download the certificate file, then go into your browser certificate settings and add the new certificate. In Google Chrome you click the little 3-dot menu in the top right corner, then settings in the drop-down menu, then Privacy and security on the left, then Security in the middle, then scroll down to the Advanced section and click on Manage certificates, then the Authorities tab, and finally the Import button and select the certificate file you just downloaded (jvastai_root.cer).I’m deleting files in Juypter but it’s not freeing disk space! How do I truly delete?
By design/default the delete button in Jupyter does not actually delete files, it just moves them to the Trash folder, which is located at:How do I run colab notebooks?
Just select the recommended pytorch image which has the jupyter launch mode pre-enabled. Select a GPU and start the instance. Once it loads, click on the jupyter button on the bottom right of the instance card to start jupyter. Then download the colab notebook as a .ipynb file and upload it to the instance in Jupyter. Then just click on that to run the notebook. Depending on the notebook you may need to install additional dependencies with apt-get or pip. As of now we don’t have a recommended colab-emulating docker image.I’m getting some missing library or package error?
Depending on the notebook, you may have to install additional dependencies. You can do this by opening a terminal in jupyter and then using regular apt-get install PACKAGE or pip install PACKAGE.How can I more easily download many files?
Jupyter Labs supports downloading multiple files by shift click to select multiple and then right click download option. But Jupyter Notebook only supports downloading individual files, and neither supports downloading folders/directories. You can use zip to more quickly download directories and large numbers of files. First open a terminal, and then in the terminal you can install zip and use that to zip up many files into a single package:Jupyter is ok, but can I run colab directly with a vast instance?
Yes. Please follow our Colab guide.SSH
How do I connect to an SSH instance on linux/mac?
On Ubuntu or Mac, first you need to generate an rsa ssh public/private keypair using the command:How do I connect to an SSH instance from Windows?
You can use Windows subsystem for Linux and then follow the ssh instructions for Linux/Mac. But some Windows users prefer a GUI tool, so here is a quick guide to connecting to an ssh instance from Windows using Putty. Make sure you save the key in ssh rsa-2 format.What is this tmux thing? How do I create multiple bash terminals on my ssh instance?
We connect you to a tmux session by default for reliability. You can create a new bash terminal window with “ctrl+b,c” (press ctrl and b, followed by c), and switch with “ctrl+b,n”. But google “tmux cheat sheet” or “tmux guide” for more info.Ok - but how can I disable tmux?
We don’t recommend this generally as ssh (especially the proxy ssh connection) can be unstable, but if you know what you are doing and need raw ssh you can add the following to your onstart:Security
How do you protect my data from other clients?
Clients are isolated to unprivileged docker containers and only have access to their own data.How do you protect my data from providers?
There are many providers on Vast.ai, ranging from tier 4 datacenters with extensive physical and operational security down to individual hobbyists renting out a few machines in their home. Our vetted datacenter partners can provide data security similar to other large cloud providers. If data security is important for your use case, you may want to rent only from our datacenter partners. Even though our smaller community providers generally do not have datacenter-level physical or operational security, they have little to gain and much to lose from stealing customer data. It can take months for providers to accumulate trust and verified status on Vast. These verified providers are thus incentivized to maintain their reputational standing just like larger cloud providers. Hosts generally have many different clients, and there are significant costs to identifying, saving, copying, and exploiting any interesting datasets, let alone any particular client’s data. You can also roughly see the relative age of a provider by their ID.Billing
How does billing work?
Once you enter a credit card and an email address and both are verified you can then increase your credit balance using one-time payments with the add credit button. Whenever your credit balance hits zero or below, your instances will be stopped automatically, but not destroyed. You are still charged storage costs for stopped instances, so it is important to destroy instances when you are done using them. Your credit card will be automatically charged periodically to pay off any outstanding negative balance.Can you bill my card automatically so I don’t have to add credit in advance?
You can set a balance threshold to configure auto billing, which will attempt to maintain your balance above the threshold. We recommend setting a threshold around your daily or weekly spend, and then setting an balance email notification threshold around 75% of that value, so that you get notified if the auto billing fails but long before your balance depletes to zero. There is also an optional debit-mode feature which can be enabled by request for older accounts. When debit-mode is enabled, your account balance is allowed to go negative (without immediately stopping your instances).I didn’t enable debit-mode - what are these automatic charges to my card?
Your card is charged automatically regardless of whether or not you have debit-mode enabled. Instances are never free - even stopped instances have storage charges. Make sure you delete instances when you are done with them - otherwise, your card will continue to be periodically charged indefinitely.How does pricing work?
There are separate prices and charges for:- Active rental (GPU) costs
- Storage costs
- Bandwidth costs
What is the billing frequency?
Balances are updated about once every few seconds.### Why should I trust vast.ai with my credit card info?You don’t need to: Vast.ai does not see, store or process your credit card numbers, they are passed directly to Stripe (which you can verify in the javascript).Do you support PayPal? What about cryptocurrency?
We currently support major credit cards through stripe and crypto payments through Coinbase and crypto.com.Data Movement
How do I upload/download to/from my instance?
You can use the CLI copy command to copy from/to directories on a remote instance and your local machine, or to copy data between two remote instances. You can use the copy buttons in the GUI to copy data between two remote instances. The copy command uses rsync and is generally fast and efficient, subject to single link upload/download constraints. Example:How do I upload/download to/from my instance - using scp?
If you launched an ssh instance, you can copy files using scp. The default ssh connection uses a proxy and thus can be slow (in terms of latency and bandwidth). Thus we recommend only using scp over the default ssh connection for smaller transfers (less than 1 GB). For larger inbound transfers, a direct connection is recommended. Downloading from a cloud data store using wget or curl can have much higher performance. The relevant scp command syntax is:I’m getting a ConnectionResetError downloading files?
This seems to be due to bugs in the urllib3 and or requests libraries used by many Python packages. We recommend using wget to download large files - it is quite robust and recovers from errors gracefully.How can I download a Kaggle dataset using wget?
First, you need to get the raw https link. Using the Chrome browser, on the Kaggle website go to the relevant dataset or competition page and start downloading the file you want. Then cancel the download and press Ctrl+J to bring up the Chrome Downloads page. At the top is the most recent download with a name and a link under it. Right-click on the URL link and use “copy link address”. Then you can use wget with that URL as follows:Examples
Disco Diffusion
Disco diffusion is powerful free and open source AI image generator, which is easy to use on vast.ai. With the right settings and powerful GPUs, it can generate artist quality high-res images for a wide variety of subjects.
Last Tower

Crashed Ship
- The disco diffusion notebooks were created for colab, but they will run in docker on vast using the common pytorch image with some slight modifications (to install a few required libs). You can use one of our slightly modified DD notebooks (5.6, 5.4, 5.2) to get started quickly.
- Instead of the pytorch image, you can use the custom jinaai/discoart docker image. For this image we recommend adding the environment variable option
-e JUPYTER_DIR=discoart
(orJUPYTER_DIR=/
) to your docker run options (directly under the image tag). This will instruct jupyter to start in a more sensible directory rather than /app which is basically empty. If you set this env variable you’ll see the discoart.ipynb notebook file is already there in the /discoart folder, no need to upload.
apt-get install -y zip
And then to zip all of the files in the default images_out/TimeToDisco directory:
zip all_images.zip images_out/TimeToDisco/*
Note: do no use spaces in your folder names, they cause headaches on linux! Use the ’\_’ underscore instead.
For discussion/help/advice running DD on vast find us on our discord, and make sure to check out the main DD discord.