vLLM (LLM inference and serving)

AI Image Generation

A lower level organization that lives within an Endpoint. It consists of a template (with extra filters for search), a set of GPU instances (workers) created from that template, and hyperparameters.

Worker_Groups

The highest level clustering of instances for the Serverless engine, consisting of a named endpoint string, a collection of Workergroups, and hyperparameters.

Endpoints

The Vast PyWorker is a Python web server designed to run alongside a machine learning model instance, providing Vast.ai Serverless compatibility.

PyWorker

Workergroup

Endpoint

The minimum number of workers you want to keep "cold" (meaning stopped and fully loaded) when your group has no load.

min_cold_workers

The maximum number of workers your router group can have.

max_workers

Image Generation

Guides

Introduction

QuickStart

Instances Help

Billing Help

Networking

Troubleshooting

Data Movement

overview

PyTorch

AI/ML Frameworks

Langflow + Ollama

AI Agents

CUDA

GPU Programming

Linux Virtual Desktop

Linux Virtual Machines

Virtual Computing

TTS with Nari Labs Dia

AI Audio Generation

Ollama + Webui

Oobabooga (LLM webui)

Huggingface TGI with LLama3

Quantized GGUF models (cloned)

AI Text Generation

Stable Diffusion

Disco Diffusion

Video Generation

AI Video Generation

Infinity Embeddings

Text Embeddings

Blender in the Cloud

Blender Batch Rendering

3D Rendering

Mining on Bittensor

Cryptocurrency

Google Colab

Development Tools

Whisper ASR Guide

Audio-to-Text

Use Cases

Teams Overview

Edit team

Teams Quickstart

Team Creation

Teams Invitations

Teams Roles

Transfer Team Ownership

team

Overview

Guide to Taxes

Datacenter Status

Payment

Verification Stages

Clusters

hosting

Multi-Node training using Torch + NCCL

distributed-computing

Search

Instance Portal

Creating a Custom Template

Templates

Instances Guide

Volumes

Billing

Earning

Referral Program

Members

Keys

Settings

console

RTX 5 Series

Specific GPUs