Authorizations
API key must be provided in the Authorization header
Body
Name of the template
256"My Template"
Docker image path
1024"ghcr.io/huggingface/text-generation-inference:1.0.3"
Docker image tag
Docker environment variables and run arguments
1024"-p 3000:3000 -e MODEL_ARGS='--model-id TheBloke/Llama-2-7B-chat-GPTQ'"
Command to run on instance startup
2048Enable direct Jupyter connection
Enable direct SSH connection
Use JupyterLab instead of Jupyter Notebook
Type of instance runtime
jupyter, ssh, args Enable SSH access
Jupyter notebook directory
256Docker repository requiring authentication
Additional machine search filters
Recommended disk space in GB