CUDA
This guide walks you through setting up and running CUDA applications on Vast.ai's cloud platform. You'll learn how to set up a CUDA development environment, connect to your instance, and develop CUDA applications efficiently using NVIDIA's development tools.
- A Vast.ai account
- Basic familiarity with CUDA programming concepts
- Basic knowledge of Linux command line
Navigate to the Templates tab to view recommended templates.
Search for NVIDIA CUDA template if:
- You need a standard CUDA development environment
- You want pre-configured security features (TLS, authentication)
- You require Jupyter notebook integration
- You need additional development tools like Tensorboard
Make a custom CUDA template if:
- You need a specific CUDA or Python version
- You have special library requirements
- You want to minimize image size for faster instance startup
You can edit the template to use Jupyter launch mode if:
- You're behind a corporate firewall that blocks SSH
- You prefer browser-based development
- You want persistent terminal sessions that survive browser disconnects
- You need quick access without SSH client setup
- You want to combine CUDA development with notebook documentation
- You plan to switch between multiple terminal sessions in the browser
You can edit the template to use SSH launch mode if:
- You need lowest possible terminal latency
- You prefer using your local terminal emulator
- You want to use advanced terminal features like tmux
- You're doing extensive command-line development
- You need to transfer files frequently using scp or rsync
Select your desired GPU configuration based on your computational needs from the Search tab. For CUDA development, consider:
- System Requirements:
- RAM: Minimum 16GB for development tools
- Storage: 10GB is usually sufficient
- CUDA Toolkit core: ~2GB
- Development files and builds: ~3-4GB
- Room for source code and dependencies: ~4GB
- CPU: 4+ cores recommended for compilation
- Network: 100+ Mbps for remote development
Rent the GPU of your choice.
Go to Instances tab to see your instance being created. There are multiple ways to connect to your instance:
- If Jupyter launch mode is selected in your template:
- Click the "OPEN" button or "Jupyter" button on your instance card
- Access a full development environment with notebook support
- If you selected SSH launch mode:
- Click Open Terminal Access button
- Copy Direct ssh connect string contents that looks like this "ssh -p 12345 [email protected] -L 8080:localhost:8080"
- You take the ssh command and execute in your terminal in your Mac or Linux-based computer or in Powershell
- The base environment includes:
- CUDA toolkit and development tools
- Python with common ML libraries
- Development utilities (gcc, make, etc.)
- Install additional CUDA dependencies:
- Navigate to your workspace:
- Set up CUDA environment variables:
CUDA not found:
- Check if GPU is detectable: nvidia-smi
If output like "No devices were found" shows up, report the machine after clicking on the wrench icon and rent a different machine.
- Code Organization
- Keep source files in ${WORKSPACE}
- Use version control for code management
- Maintain separate directories for builds and source
- Performance Optimization
- Use proper CUDA stream management
- Optimize memory transfers
- Profile code using NVIDIA tools
Create a provisioning script for custom environment setup:
Configure VS Code or other IDEs for remote development:
- Use SSH port forwarding for secure connections
- Configure development tools to use remote CUDA compiler
- Set up source synchronization using Syncthing
You now have a fully configured CUDA development environment on Vast.ai. This setup provides the flexibility of cloud GPU resources with the convenience of local development.