Debugging

pyworker

A lower level organization that lives within an Endpoint. It consists of a template (with extra filters for search), a set of GPU instances (workers) created from that template, and hyperparameters.

Worker_Groups

The highest level clustering of instances for the autoscaler, consisting of a named endpoint string, a collection of Worker groups, and hyperparameters.

Endpoints

The Vast PyWorker is a Python web server designed to run alongside a machine learning model instance, providing autoscaler compatibility.

PyWorker

Worker_Group

Endpoint

The minimum number of workers you want to keep "cold" (meaning stopped and fully loaded) when your group has no load.

min_cold_workers

The maximum number of workers your router group can have.

max_workers

Overview

Serverless

Getting Started

Architecture

Inside an Autoscaler GPU

Creating New PyWorkers

Performance Testing

Route

Logs

Worker List

endpoints

Autoscaler

Text Generation Inference (TGI)

Comfy UI

Untitled

Python webserver that collects metrics for the autoscaler.

Pyworker