> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vast.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Comfy UI
> Learn how to use Comfy UI with Vast.ai Serverless for image generation workflows.
The [ComfyUI Serverless template](https://cloud.vast.ai/?ref_id=62897\&creator_id=62897\&name=ComfyUI%20\(Serverless\)) allows you to send requests to ComfyUI and have generated assets automatically uploaded to S3-compatible storage. The template returns pre-signed URLs in response to requests, along with detailed process updates emitted by ComfyUI during generation.
# Template Components
The ComfyUI template includes:
**In the Docker Image:**
* ComfyUI
* ComfyUI API Wrapper
* Stable Diffusion 1.5 (for benchmarking)
**Downloaded on first boot:**
* PyWorker (comfyui-json worker)
* Provisioning Script for custom configuration
* Adds cron job to remove older output files (older than 24 hours) if available disk space is less than 512MB
Before using this template, familiarize yourself with the [Serverless Documentation](/guides/serverless/overview) and [Serverless Quickstart](/guides/serverless/quickstart).
# Environment Variables
## Required for S3 Storage
The API wrapper manages asset uploads to S3-compatible storage. Configure these variables in your [Account Settings](https://cloud.vast.ai/settings/):
* `S3_ACCESS_KEY_ID`(string): Access key ID for S3-compatible storage
* `S3_SECRET_ACCESS_KEY`(string): Secret access key for S3-compatible storage
* `S3_BUCKET_NAME`(string): Bucket name for S3-compatible storage
* `S3_ENDPOINT_URL`(string): Endpoint URL for S3-compatible storage
* `S3_REGION`(string): Optional region for S3-compatible storage
These S3 values can be overridden on a per-request basis in the request payload.
## Optional Configuration
* `WEBHOOK_URL`(string): Optional webhook to call after generation completion or failure
* `PYWORKER_REPO`(string): Custom PyWorker git repository URL (default: [https://github.com/vast-ai/pyworker](https://github.com/vast-ai/pyworker))
* `PYWORKER_REF`(string): Git reference to checkout from PyWorker repository
* `BENCHMARK_TEST_WIDTH`(int): Image width in pixels for default benchmark only (default: 512)
* `BENCHMARK_TEST_HEIGHT`(int): Image height in pixels for default benchmark only (default: 512)
* `BENCHMARK_TEST_STEPS`(int): Number of denoising steps for default benchmark only (default: 20)
Store sensitive information like API keys in the 'Environment Variables' section of your [Account Settings](https://cloud.vast.ai/settings/). These will be available in all instances you create.
# Benchmarking
When a worker initializes, it runs a benchmark to validate GPU performance and calculate a performance score. This score determines how requests are distributed across workers.
## Custom Benchmark Workflows
You can provide a custom ComfyUI workflow for benchmarking by creating `workers/comfyui-json/misc/benchmark.json`. Use the placeholder `__RANDOM_INT__` in place of static seed values to ensure varied generations.
The `__RANDOM_INT__` placeholder can also be used in any workflow you send to the worker. It will be replaced with a random integer for each generation, allowing for varied outputs without manually specifying different seeds.
## Default Benchmark
If no custom benchmark is provided, the template uses Stable Diffusion v1.5 with ComfyUI's default text-to-image workflow. Configure the benchmark complexity using the environment variables listed above.
**How worker scores work:**
* Each benchmark has a baseline complexity score of `100` (representing 100% of the work)
* A worker completing the benchmark in 100 seconds receives a score of `1.0` (processes 1% of work per second)
* A worker completing in 50 seconds receives a score of `2.0` (twice as fast)
* Faster workers (higher scores) receive proportionally more requests
Configure your benchmark to match your actual workload complexity. If your typical workflow is more complex than the default SD1.5 benchmark, increase `BENCHMARK_TEST_STEPS` proportionally.
# Endpoints
The ComfyUI template provides endpoints for executing workflows and generating images. After obtaining a worker address from the `/route/` endpoint (see [route documentation](/guides/serverless/architecture)), you can send requests to the following endpoints.
## /generate/sync
The primary endpoint for submitting ComfyUI workflows. This endpoint accepts complete, user-defined ComfyUI workflows in JSON format and processes them synchronously.
### Request Structure
### Input
`payload`:
* `input`:
* `request_id`(string): Optional unique identifier for tracking the request
* `workflow_json`(object): Complete ComfyUI workflow graph in JSON format
* `s3`(object): Optional S3 configuration override
* `access_key_id`(string)
* `secret_access_key`(string)
* `endpoint_url`(string)
* `bucket_name`(string)
* `region`(string)
* `webhook`(object): Optional webhook configuration
* `url`(string): Webhook URL to call after generation
* `extra_params`(object): Additional parameters to include in webhook payload
If the API Wrapper detects that you have used a URL as an input image in your workflow, it will automatically download the image and modify the workflow to use the local path.
**Example Request:**
```python vastai icon=python theme={null}
from vastai import Serverless
import asyncio
ENDPOINT_NAME="comfyui-json"
async def main():
async with Serverless() as client:
endpoint = await client.get_endpoint(name=ENDPOINT_NAME)
# ComfyUI API compatible json workflow
workflow = {
"3": {
"inputs": {
"seed": "__RANDOM_INT__",
"steps": 20,
"cfg": 8,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 1,
"model": ["4", 0],
"positive": ["6", 0],
"negative": ["7", 0],
"latent_image": ["5", 0]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"4": {
"inputs": {
"ckpt_name": "v1-5-pruned-emaonly-fp16.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
},
"5": {
"inputs": {
"width": 512,
"height": 512,
"batch_size": 1
},
"class_type": "EmptyLatentImage",
"_meta": {
"title": "Empty Latent Image"
}
},
"6": {
"inputs": {
"text": "beautiful scenery nature glass bottle landscape, purple galaxy bottle",
"clip": ["4", 1]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"7": {
"inputs": {
"text": "text, watermark",
"clip": ["4", 1]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"8": {
"inputs": {
"samples": ["3", 0],
"vae": ["4", 2]
},
"class_type": "VAEDecode",
"_meta": {
"title": "VAE Decode"
}
},
"9": {
"inputs": {
"filename_prefix": "ComfyUI",
"images": ["8", 0]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
}
}
payload = {
"input": {
"request_id": "",
"workflow_json": workflow,
"s3": {
"access_key_id": "",
"secret_access_key": "",
"endpoint_url": "",
"bucket_name": "",
"region": ""
},
"webhook": {
"url": "",
"extra_params": {
"user_id": "12345",
"project_id": "abc-def"
}
}
}
}
response = await endpoint.request("/generate/sync", payload)
# Response contains status, output, and any errors
print(response["response"])
if __name__ == "__main__":
asyncio.run(main())
```
### Outputs
* `id`(string): Unique identifier for the request
* `status`(string): Request status - `completed`, `failed`, `processing`, `generating`, or `queued`
* `message`(string): Human-readable status message
* `comfyui_response`(object): Detailed response from ComfyUI including:
* `prompt`: The workflow that was executed
* `outputs`: Generated outputs organized by node ID
* `status`: Execution status with completion messages and timestamps
* `meta`: Metadata about the execution
* `execution_details`: Progress updates and timing information
* `output`(array): Array of output objects, each containing:
* `filename`(string): Name of the generated file
* `local_path`(string): Path to file on worker
* `url`(string): Pre-signed URL for downloading the generated asset (if S3 is configured)
* `type`(string): Output type (e.g., "output")
* `subfolder`(string): Subfolder within output directory
* `node_id`(string): ComfyUI node that produced this output
* `output_type`(string): Type of output (e.g., "images")
* `timings`(object): Timing information for the request
Ensure that any models and nodes referenced in your workflow are already installed before sending a request. Use the Provisioning Script or build a custom Docker image to pre-install required models and nodes.
# Testing Before Deployment
Although this template is designed for serverless deployment, you can test it as an interactive instance first.
To access ComfyUI and the API wrapper:
1. Start an interactive instance with this template
2. Connect via SSH with port forwarding:
```bash theme={null}
ssh -L 18188:localhost:18188 -L 18288:localhost:18288 root@instance-address -p port
```
3. Access ComfyUI at [http://localhost:18188](http://localhost:18188)
4. Access the API Wrapper documentation at [http://localhost:18288/docs](http://localhost:18288/docs)
The benchmarking process will be visible in the instance logs, but applications won't be available over the public interface without port forwarding.