Cloud Backups

comprehensive guide to cloud backups on vast ai introduction this guide walks you through setting up and running automated backups for your vast ai container instances to cloud storage cloud backups can you help preserve your work when using vast's docker based instances with proper backup strategies, you can ensure your valuable data remains safe and accessible even if your instance goes offline prerequisites a vast ai account access to a vast ai docker based instance cloud storage connection set up in vast ai (optional) install and use vast cli (optional) understanding of how to use cron in computers with unix like os setup 1\ setting up cloud storage connections before creating backup jobs, you need to ensure you have a cloud storage connection set up in your vast ai account you can view your existing connections using the vast cli python3 vast py show connections id name cloud type 19447 karthik vast ai google drive drive if you don't have a connection yet, you'll need to set one up in vast ai's settings page before proceeding with backup operations 2\ understanding backup options vast ai provides multiple approaches to schedule data backups using vast's job scheduling system via cli create hourly, daily, or weekly automated backup jobs using cron on your personal computer schedule backups with custom timing from your local machine both approaches have their advantages depending on your workflow and requirements backup methods 1\ using cli for scheduled backups the vast cli tool allows you to create scheduled backup jobs with several timing options the basic structure of a scheduled backup command includes these parameters \ schedule schedule values hourly, daily, weekly \ start date start date start date in format 'yyyy mm dd hh\ mm\ ss pm' (utc) \ end date end date end date in format 'yyyy mm dd hh\ mm\ ss pm' (utc) \ day day day of week (0 6, where 0=sunday) or " " \ hour hour hour of day (0 23) or " " you can run this command to see more details about these parameters python3 vast py cloud copy help let's explore the different scheduling options to create a weekly backup job that runs every saturday at 9 pm utc python3 vast py cloud copy src /workspace dst /backups/19015821 backups/ instance 19015821 connection 19447 transfer "instance to cloud" schedule weekly day 6 hour 21 in this command \ src /workspace specifies the source directory on your instance dst /backups/19015821 backups/ is the destination folder in your cloud storage instance 19015821 is your instance's id connection 19447 is your cloud storage connection id day 6 represents saturday (0=sunday, 1=monday, etc ) hour 21 represents 9 pm utc (0=12am utc, 1=1am utc, etc ) for daily backups at a specific hour (e g , 9 pm utc every day) python3 vast py cloud copy src /workspace dst /backups/19015821 backups/ instance 19015821 connection 19447 transfer "instance to cloud" schedule daily day " " hour 21 the day " " parameter indicates that the job should run every day for hourly backups that run every hour of every day python3 vast py cloud copy src /workspace dst /backups/19015821 backups/ instance 19015821 connection 19447 transfer "instance to cloud" schedule hourly day " " hour " " setting both day " " and hour " " along with schedule hourly makes the job run every hour to update your backup schedule, simply run the same command with the new schedule the system will prompt you for confirmation, and upon acceptance, it will update the schedule accordingly existing scheduled job found do you want to update it (y|n)? y add scheduled job update success scheduling daily job to cloud copy from 1745599087 0 to 1746599887 0 2\ using cron on your personal linux computer if you prefer more granular control over your backup schedule, you can use cron on your local linux or mac computer this approach allows for customized schedules beyond the hourly/daily/weekly options first, open your crontab file for editing crontab e then, add a line that specifies your backup schedule for example, to run a backup every 4 hours 0 /4 python3 vast py cloud copy src /workspace dst /backups/19015821 backups/ instance 19015821 connection 19447 transfer 'instance to cloud' in this cron schedule 0 represents the minute (0th minute of the hour) /4 means "every 4 hours" the three asterisks represent day of month, month, and day of week, indicating "every day" viewing scheduled backup jobs to see all your currently scheduled backup jobs python3 vast py show scheduled jobs example output scheduled job id instance id api endpoint start (date/time in utc) end (date/time in utc) day of the week hour of the day in utc minute of the hour frequency 1 19778412 /api/v0/commands/rclone/ 2025 04 24/23 38 2028 05 06/23 38 everyday 4 pm 00 daily 2 19782577 /api/v0/commands/rclone/ 2025 04 29/23 47 2025 05 09/23 47 wednesday 10 am 00 weekly 3 19757389 /api/v0/commands/rclone/ 2025 05 01/00 04 2026 05 01/00 04 everyday every hour 00 hourly understanding the output field description scheduled job id unique identifier for your job (needed for deletion) instance id the instance this job is associated with api endpoint the endpoint being called (rclone is used for backups to cloud storage) start (date/time) start date/time of period when this scheduled job will be executed (in utc) end (date/time) end date/time of period when this scheduled job will be executed (in utc) default is the end of the contract day of the week which day the job runs (can be specific day like "wednesday", "saturday", or "everyday") hour of the day at what hour the job runs (formatted as 1 pm, 11 pm, 8 pm in utc, etc ) minute of the hour at what minute of the specified hour the job runs (00, 33, 10, etc ) frequency how often the job runs (hourly, daily, weekly) examples explained job 1 a daily backup that runs every day at 4 00 pm utc runs daily at the same time will continue running from apr 24, 2025 until may 6, 2028 job 2 a weekly backup that runs on wednesdays at 10 00 am utc runs only on wednesdays at 10 00 am utc short duration job (apr 29 may 9, 2025) job 3 a hourly backup that runs every hour of every day runs every hour (1 am, 2 am, 3 am, etc ) will continue running for a year deleting scheduled backup jobs if you need to remove a scheduled backup job that you no longer want to run, you can use the delete scheduled job command followed by the job id bash python3 vast py delete scheduled job job id for example python3 vast py delete scheduled job 4462309 this will completely remove the scheduled job from the system when successful, you'll receive a confirmation message {'success' true, 'msg' 'scheduled job 4462309 deleted successfully'} find job ids to delete to find the id of the job you want to delete, first run python3 vast py show scheduled jobs you'll see output similar to scheduled job id instance id api endpoint start (date/time) end (date/time) day of the week hour of the day minute of the hour frequency 4462317 19281511 /api/v0/commands/rclone/ 2025 04 08/09 01 2028 06 08/18 48 everyday 1 pm 33 daily 4462321 19489711 /api/v0/commands/rclone/ 2025 04 15/20 00 2025 04 19/20 00 saturday 11 pm 00 weekly 4462322 19490133 /api/v0/commands/rclone/ 2025 04 15/20 00 2025 04 19/20 00 wednesday 8 pm 10 weekly the scheduled job id column in the output contains the ids you'll need for deletion best practices choose the right backup frequency consider these factors when determining how often to back up your data how frequently your data changes the criticality of your data the cost of data loss the performance impact of backup operations the bandwidth costs of backing your data up in cloud storage back up only what you need be selective about what you back up to save time and storage costs focus on backing up only important data (models, results, custom code) verify your backups periodically check that your backups are working correctly download a sample backup from cloud storage and verify its contents check logs for any cloud copy failures test the restoration process before you actually need it if contract is extended, update end date of scheduled job conclusion setting up regular backups for your vast ai instances can be a valuable part of a robust workflow by choosing the appropriate backup method and schedule, you can ensure that your valuable work remains safe and accessible regardless of instance lifecycle events remember that the best backup system is one that you set up before you need it take time now to implement a backup strategy that meets your needs, and you can thank yourself later additional resources vast ai documentation https //vast ai/docs/ vast ai cli repository https //github com/vast ai/vast python cron job documentation https //en wikipedia org/wiki/cron