Skip to main content
The Comfy UI serverless template can be used to run Comfy stable diffusion models on Vast GPU instances. This page documents required environment variables and endpoints to get started. A full PyWorker and Client implementation can be found here.

​
Environment Variables

  • HF_TOKEN(string): HuggingFace API token with read permissions, used to download gated models. Read more about HuggingFace tokens here.
  • COMFY_MODEL(string): Text2Image model being used.
    • options: "sd3", "flux"

​
Endpoints

​
/prompt/

The ComfyUI defined endpoint to generate an image. After receving a GPU worker address from the /route/ endpoint, a /prompt/ endpoint would have the following structure.

​
Inputs

For both “sd3” and “flux”: Payload:
  • prompt(string): The model prompt used to generate the image.
  • width(int): Width in pixels of the resulting generated image.
  • height(int): Height in pixels of the resulting generated image.
  • steps(int): Number of diffusion steps taken during image generation. The assumed default is 28.
  • seed(int): The value used to initialize the random noise pattern in the diffusion model. Used for reproducability.
Auth_data:
  • signature(string): A cryptographic string that authenticates the url, cost, and reqnum fields in the response, proving they originated from the server. Clients can use this signature, along with the server’s public key, to verify that these specific details have not been tampered with.
  • cost(float): The estimated compute resources for the request. The units of this cost are defined by the PyWorker.
  • endpoint(string): Name of the Endpoint.
  • reqnum(int): The request number corresponding to this worker instance. Note that workers expect to receive requests in approximately the same order as these reqnums, but some flexibility is allowed due to potential out-of-order requests caused by concurrency or small delays on the proxy server.
  • url(string): The address of the worker instance to send the request to.
JSON
{
  "payload": {
    "prompt": "a majestic lion with a flowing mane, fantasy art style",
    "width": 1024,
    "height": 1024,
    "steps": 28,
    "seed": 55512345
  },
  "auth_data": {
    "signature": "mock_signature_from_routing_service_abc123",
    "cost": 100,
    "endpoint": "comfyui-flux-default",
    "reqnum": "req_789xyz",
    "url": "http://127.0.0.1:8188/" //this is the base URL of the worker, 'prompt/' is appended to the end
  }
}

​
Outputs

  • prompt_id(string): A unique identifier assigned by the ComfyUI instance for this specific workflow execution or prompt.
  • number(int): A sequential number assigned by the ComfyUI instance to this prompt. This number reflects the order in which prompts are received or processed by the worker.
  • node_errors(object): An object that contains details about any errors that occurred within specific nodes during the workflow execution.
  • outputs(object): An object containing the outputs generated by specific nodes in the ComfyUI workflow. The keys of this object are the string identifiers of the nodes that produced output
JSON
{
  "prompt_id": "f9e0a3c2-0b7a-4f8e-8c1d-5a7b9e2f3c0a",
  "number": 7,
  "node_errors": {},
  "outputs": {
    "9": {
      "images": [
        {
          "filename": "ComfyUI_00007_.png",
          "subfolder": "",
          "type": "output"
        }
      ]
    }
  }
}

​
/custom_workflow/

Allows a client to send a complete, user-defined ComfyUI workflow (in JSON format) to the server for execution. It includes the payload and auth_data objects, but modified according to the user’s design. The workflow is a direct ComfyUI graph. Here is an example, similar to the SD3 workflow.
JSON
{
  "payload": {
    "custom_fields": {
      "steps": 20,
      "width": 512,
      "height": 512
    },
    "workflow": {
      "3": {
        "inputs": {
          "seed": 156680208700286,
          "steps": 20,
          "cfg": 8,
          "sampler_name": "euler",
          "scheduler": "normal",
          "denoise": 1,
          "model": ["4", 0],
          "positive": ["6", 0],
          "negative": ["7", 0],
          "latent_image": ["5", 0]
        },
        "class_type": "KSampler"
      },
      "4": {
        "inputs": {
          "ckpt_name": "sd3_medium_incl_clips_t5xxlfp16.safetensors"
        },
        "class_type": "CheckpointLoaderSimple"
      },
      "5": {
        "inputs": {
          "width": 512,
          "height": 512,
          "batch_size": 1
        },
        "class_type": "EmptyLatentImage"
      },
      "6": {
        "inputs": {
          "text": "a futuristic cityscape at dusk, neon lights, flying vehicles",
          "clip": ["4", 1]
        },
        "class_type": "CLIPTextEncode"
      },
      "7": {
        "inputs": {
          "text": "blurry, low quality, ugly",
          "clip": ["4", 1]
        },
        "class_type": "CLIPTextEncode"
      },
      "8": {
        "inputs": {
          "samples": ["3", 0],
          "vae": ["4", 2]
        },
        "class_type": "VAEDecode"
      },
      "9": {
        "inputs": {
          "filename_prefix": "CustomWorkflowOutput",
          "images": ["8", 0]
        },
        "class_type": "SaveImage"
      }
    }
  },
  "auth_data": {
    "signature": "mock_signature_for_custom_workflow_def456",
    "cost": 150,
    "endpoint": "comfyui-sd3-custom",
    "reqnum": "req_101112jkl",
    "url": "http://127.0.0.1:8188/"
  }
}