Linux Virtual Machines

complete guide to running virtual machines on vast ai introduction vast ai provides virtual machine (vm) capabilities alongside its docker based instance rentals this guide walks you through everything you need to know about running vms on machines with gpus found at vast prerequisites a vast ai account ssh client installed on your local machine and ssh public key added the keys section at cloud vast ai (optional) install and use vast cli vm vs docker understanding the differences vm advantages full support for init managers like systemd enable running docker, kubernetes, and snap applications perfect for containerization within your instance process tracing support via ptrace ideal for debugging and system monitoring complete system isolation full control over the virtual environment vm limitations longer instance creation and boot times compared to docker higher disk space requirements limited machine selection fewer preconfigured templates currently restricted to ssh only launch mode common use cases deep learning development environments custom ml framework setups run multiple ml framework versions simultaneously with full control over cuda versions, perfect for maintaining compatibility with legacy projects while using newer frameworks distributed training systems set up complete kubernetes clusters for distributed machine learning, enabling efficient training of large models across multiple nodes development and testing docker compose development deploy and test multi container applications with full docker compose support, including volume mounts and network configurations not possible in regular docker instances cuda performance profiling profile cuda applications with full system access and hardware counters, enabling detailed performance analysis and optimization containerization development test docker and kubernetes configurations in fully isolated environments with docker in docker capabilities system level development develop and test custom drivers and kernel modules with direct access to system resources production workloads database systems run database servers with full control over system parameters and storage configurations for optimal performance web services deploy web applications requiring specific system level configurations or systemd integration research and academic use reproducible research create and preserve complete system environments to ensure research reproducibility across different setups gpu architecture research conduct low level gpu research with direct hardware access and custom driver configurations security testing isolated security research perform security testing and malware analysis in completely isolated environments without risking host system contamination legacy application support legacy software run older applications that require specific operating system versions or library combinations not available in containers resource intensive applications high performance computing configure custom parallel computing environments with specific network and scheduler requirements graphics and rendering set up rendering systems with precise control over gpu configurations and driver versions getting started prerequisites ssh key setup (required) note you must add your ssh public key to the keys section after logging into your vast ai account before creating a vm instance if you start a vm before any ssh keys have been added to your account, the vm will not be accessible steps to setup your ssh key generate an ssh key and copy your public key access your account settings page navigate to the ssh keys section add your public key note ssh keys cannot be modified once a vm is running creating and configuring vms search for ubuntu vm template go to templates tab and search for recommended ubuntu 22 04 vm template edit template as needed when you find the ubuntu 22 04 vm template, you can edit the template environment variables you can set environment variables by editing the vm template and adding a specific environment variable name and value in the environment variables section or adding a line like this to "docker options" field docker e jupyter dir=/ e test=ok variables are written to /etc/environment to use these environment variables in a script once you're inside your machine, run this command linux source /etc/environment expose ports publicly you can expose ports publicly by editing the vm template and adding specific ports in ports section or adding a line similar to this in "docker options" field docker p 8081 8081 p 8082 8082/udp p 70000 70000 specify on start script configuration the on start script field allows specifying a script to run on instance start unlike in docker based instances, the interpreter must be specified by a shebang here's an example for bash linux #!/bin/bash rent a machine rent a machine of your choice from the search tab you can see the instance being created in the instances tab it can take some time to load connect to your vm once the blue button the instance card says > connect , you can click the button and copy the ssh command to execute in your terminal in your mac or linux based computer you can also use powershell or windows putty tools if you have a windows computer vm management cloud copy utility different from docker based instances use cli command vastai vm copy $src vm id $dest vm id limitations only supports full vm migration copying between vms only (no external storage support) no individual folder copy support best practices resource management monitor disk usage due to higher overhead plan for longer boot times in your workflows security keep ssh keys secure configure firewall rules appropriately regular security updates performance optimization use appropriate vm sizes for your workload monitor resource utilization clean up unused resources troubleshooting common issues vm won't start check if ssh key is added in account verify that rented machine supports vms environment variables not working ensure variables are properly set in docker options check if /etc/environment is being sourced verify script permissions connectivity issues verify ssh key permissions check network configuration confirm port forwarding setup try a different host machine support resources vast ai documentation vast ai discord support chat at vast ai conclusion virtual machines on vast ai provide powerful capabilities for specific use cases, particularly those requiring full system control or containerization support while they have some limitations compared to docker instances, their flexibility and isolation make them ideal for many advanced computing scenarios