Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.vast.ai/llms.txt

Use this file to discover all available pages before exploring further.

Search for benchmark results using custom query

​
Usage

vastai search benchmarks [--help] [--api-key API_KEY] [--raw] <query>

​
Arguments

​
query
string
required
Search query in simple query syntax (see below)

​
Description

Query syntax 
query      = comparison comparison...
comparison = field op value
field      = <name of a field>
op         = one of: <, <=, ==, !=, >=, >, in, notin
value      = <bool, int, float, string> | 'any' | [value0, value1, ...]
bool       = True | False
  • Wrap > and < in quotes on the command line
  • For string values (e.g. gpu_name), replace spaces with underscores: RTX_3090 not RTX 3090

​
Examples

# search for benchmarks with score > 100 for llama2_70B model on 2 specific machines
vastai search benchmarks 'score > 100.0  model=llama2_70B  machine_id in [302,402]'

​
Available Fields

FieldTypeDescription
contract_idintID of instance/contract reporting benchmark
idintbenchmark unique ID
imagestringimage used for benchmark
last_updatefloatdate of benchmark
machine_idintid of machine benchmarked
modelstringname of model used in benchmark
namestringname of benchmark
num_gpusintnumber of gpus used in benchmark
scorefloatbenchmark score result

​
Global Options

The following options are available for all commands:
OptionDescription
--url URLServer REST API URL
--retry NRetry limit
--rawOutput machine-readable JSON
--explainVerbose explanation of API calls
--api-key KEYAPI key (defaults to ~/.config/vastai/vast_api_key)