Signature
Parameters
Optional[float]
default:"0.0"
minimum floor load in perf units/s (token/s for LLms)
Optional[float]
default:"0.0"
minimum floor load in perf units/s (token/s for LLms), but allow handling with cold workers
Optional[float]
default:"0.9"
target capacity utilization (fraction, max 1.0, default 0.9)
Optional[float]
default:"2.5"
cold/stopped instance capacity target as multiple of hot capacity target (default 2.5)
Optional[int]
default:"5"
min number of workers to keep ‘cold’ when you have no load (default 5)
Optional[int]
default:"20"
max number of workers your endpoint group can have (default 20)
Optional[str]
deployment endpoint name (allows multiple autoscale groups to share same deployment endpoint)
Optional[float]
maximum seconds requests may be queued on each worker (default 30.0)
Optional[float]
target seconds for the queue to be cleared (default 10.0)
Optional[int]
seconds of no traffic before the endpoint can scale to zero active workers
Returns
dict