Multi-Node training using Torch + NCCL

console

A lower level organization that lives within an Endpoint. It consists of a template (with extra filters for search), a set of GPU instances (workers) created from that template, and hyperparameters.

Worker_Groups

The highest level clustering of instances for the Serverless engine, consisting of a named endpoint string, a collection of Workergroups, and hyperparameters.

Endpoints

The Vast PyWorker is a Python web server designed to run alongside a machine learning model instance, providing Vast.ai Serverless compatibility.

PyWorker

Workergroup

Endpoint

The minimum number of workers you want to keep "cold" (meaning stopped and fully loaded) when your group has no load.

min_cold_workers

The maximum number of workers your router group can have.

max_workers

Introduction

Guides

QuickStart

Instances Help

Billing Help

Networking

Troubleshooting

Data Movement

overview

PyTorch

AI/ML Frameworks

Langflow + Ollama

AI Agents

CUDA

GPU Programming

Linux Virtual Desktop

Linux Virtual Machines

Virtual Computing

TTS with Nari Labs Dia

AI Audio Generation

Ollama + Webui

Oobabooga (LLM webui)

Huggingface TGI with LLama3

Quantized GGUF models (cloned)

vLLM (LLM inference and serving)

AI Text Generation

Image Generation

Stable Diffusion

Disco Diffusion

AI Image Generation

Video Generation

AI Video Generation

Infinity Embeddings

Text Embeddings

Blender in the Cloud

Blender Batch Rendering

3D Rendering

Mining on Bittensor

Cryptocurrency

Google Colab

Development Tools

Whisper ASR Guide

Audio-to-Text

use-cases

Teams Overview

Edit team

Teams Quickstart

Team Creation

Teams Invitations

Teams Roles

Transfer Team Ownership

team

Overview

Guide to Taxes

Datacenter Status

Payment

Verification Stages

Clusters

hosting

distributed-computing

Search

Instance Portal

Creating a Custom Template

Templates

Instances Guide

Volumes

Billing

Earning

Referral Program

Members

Keys

Settings

RTX 5 Series

Specific GPUs