- Instance<->Instance and Instance<->Local copy using the
vastai copy
CLI command - Instance<->Instance copy in the GUI instance control panel or
vastai copy
CLI command - Instance<->Cloud copy using the GUI instance control panel or
vastai cloud copy
CLI command
- Instance<->Instance migration through the
vastai vm copy
CLI command or the GUI instance control panel
Instance<->Cloud copy (cloud sync) (Docker)
The cloud sync feature allows you to copy data to/from instance local storage and several cloud storage providers (S3, gdrive, backblaze, etc) - even when the instance is stopped.Using the GUI
Vast currently supports Dropbox, Amazon S3 and Backblaze cloud storage providers. First you will need to connect to the cloud provider on the account page and then use the cloud copy button on the instance to start the copy operation.Cloud Copy
Using the CLI
You can also access this feature using thevastai cloud copy
CLI command.
Instance <-> Instance copy
Instance to instance copy allows moving data directly between the local storage of two instances. If the two instances are on the same machine or the same local network (same provider and location) then the copy can run at faster local network storage speeds and there is no internet transit cost.Using the GUI
You can use the copy buttons to copy data between two instances. Instances can be stopped/inactive. See complete Constraints below. Click the copy button on the source instance and then on the destination instance to bring up the copy dialogue. For docker-based instances you will see the following folder dialogue.
Itoicopy
WARNING
You should not copy to /root or / as a destination directory, as this can mess up the permissions on your instance ssh folder, breaking future copy operations (as they use ssh authentication).
You should not copy to /root or / as a destination directory, as this can mess up the permissions on your instance ssh folder, breaking future copy operations (as they use ssh authentication).
Copy Modal
Using the CLI
You can also access this feature using thevastai copy
CLI command.
CLI Copy Command (Docker)
You can use the CLI copy command to copy from/to directories on a remote instance and your local machine, or to copy data between two remote instances. The copy command uses rsync and is generally fast and efficient, subject to single link upload/download constraints. Example:Text
CLI Copy Command (VMs)
You can use the CLI vm copy command to copy your entire VM from one instance to another. The destination VM’s disk will be replaced with the contents of the source machine. Example:Text
Constraints
For VM-based instances, the destination instance must be stopped during the transfer.WARNING
You should not copy to /root or / as a destination directory, as this can mess up the permissions on your instance ssh folder, breaking future copy operations (as they use ssh authentication).
You should not copy to /root or / as a destination directory, as this can mess up the permissions on your instance ssh folder, breaking future copy operations (as they use ssh authentication).
Performance
If your data is already stored in the cloud (S3, gdrive, etc) then you should naturally use the appropriate linux CLI or commands to download and upload data directly, or you could use the cloud sync feature. This generally will be one the fastest methods for moving large quantities of data, as it can fully saturate a large number of download links. If you are using multiple instances with significant data movement requirements you will want to use high bandwidth cloud storage to avoid any single machine bottlenecks. If you launched a Jupyter notebook instance, you can use its upload feature, but this has a file size limit and can be slow. You can also use standard Linux tools like scp, ftp, rclone, or rsync to move data. For moving code and smaller files scp is fast enough and convenient. However, be warned that the default ssh connection uses a proxy and can be slow for large transfers (direct ssh recommended). Instance to instance copy is generally as fast as other methods, and can be much faster (and cheaper) for moving data between instances on the same datacenter.SCP
If you launched an ssh instance, you can copy files using scp. The proxy ssh connection can be slow (in terms of latency and bandwidth). Thus we recommend only using scp over the proxy ssh connection for smaller transfers (less than 1 GB). For larger inbound transfers, using the direct ssh connection is recommended. Downloading from a cloud data store using wget or curl can have much higher performance. The relevant scp command syntax is:Text
Text
Text
Text