​

CUDA Programming on Vast.ai

​

Introduction

​

Prerequisites

• A Vast.ai account
• Basic familiarity with CUDA programming concepts
• Basic knowledge of Linux command line
• (Optional) Install TLS Certificate for Jupyter
• (Optional) SSH client installed on your local machine and SSH public key added the Keys section at cloud.vast.ai
• (Optional) Vast-cli installed on your local machine for command-line management
• (Optional) Docker knowledge for customizing development environments

​

Setup

​

1. Selecting the Right Template

• You need a standard CUDA development environment
• You want pre-configured security features (TLS, authentication)
• You require Jupyter notebook integration
• You need additional development tools like Tensorboard

• You need a specific CUDA or Python version
• You have special library requirements
• You want to minimize image size for faster instance startup

​

2. Edit the Template and Select Template

• You’re behind a corporate firewall that blocks SSH
• You prefer browser-based development
• You want persistent terminal sessions that survive browser disconnects
• You need quick access without SSH client setup
• You want to combine CUDA development with notebook documentation
• You plan to switch between multiple terminal sessions in the browser

• You’re using VSCode Remote-SSH or other IDE integrations
• You need lowest possible terminal latency
• You prefer using your local terminal emulator
• You want to use advanced terminal features like tmux
• You’re doing extensive command-line development
• You need to transfer files frequently using scp or rsync

​

2. Create Your Instance

• System Requirements:
  • RAM: Minimum 16GB for development tools
  • Storage: 10GB is usually sufficient
    • CUDA Toolkit core: ~2GB
    • Development files and builds: ~3-4GB
    • Room for source code and dependencies: ~4GB
  • CPU: 4+ cores recommended for compilation
  • Network: 100+ Mbps for remote development

​

3. Connecting to Your Instance

• If Jupyter launch mode is selected in your template:
  • Click the “OPEN” button or “Jupyter” button on your instance card
  • Access a full development environment with notebook support
• If you selected SSH launch mode:
  • Click Open Terminal Access button
  • Copy Direct ssh connect string contents that looks like this “ssh -p 12345 root@123.456.789.10 -L 8080:localhost:8080”
  • You take the ssh command and execute in your terminal in your Mac or Linux-based computer or in Powershell
  • You can use Powershell or Windows Putty tools if you have a Windows computer

​

Installation

​

Setting Up Your Development Environment

1. The base environment includes:
   • CUDA toolkit and development tools
   • Python with common ML libraries
   • Development utilities (gcc, make, etc.)
2. Install additional CUDA dependencies:

apt-get update
apt-get install -y cuda-samples

​

Configuring Your Workspace

1. Navigate to your workspace:

cd ${WORKSPACE}

1. Set up CUDA environment variables:

echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

​

Troubleshooting

​

Common Issues and Solutions

• Check if GPU is detectable: nvidia-smi

nvidia-smi

​

Best Practices

​

Development Workflow

• Code Organization
  • Keep source files in ${WORKSPACE}
  • Use version control for code management
  • Maintain separate directories for builds and source
• Performance Optimization
  • Use proper CUDA stream management
  • Optimize memory transfers
  • Profile code using NVIDIA tools

​

Advanced Topics

​

Custom Environment Setup

#!/bin/bash
. /venv/main/bin/activate
pip install additional-packages
wget custom-tools.tar.gz

​

Remote Development Setup

• Use SSH port forwarding for secure connections
• Configure development tools to use remote CUDA compiler
• Set up source synchronization using Syncthing

​

Conclusion

​

Additional Resources

• NVIDIA CUDA Documentation
• Vast.ai Documentation
• CUDA Sample Projects