Disco Diffusion

deprecated please see stable diffusion guide overview disco diffusion is an incredibly powerful free and open source ai image generator, which is easy to use on vast ai with the right settings and powerful gpus, it can generate artist quality high res images for a wide variety of subjects all of these images were generated purely through dd on vast ai, without any other tools or clean up there are a few ways to run disco diffusion on vast the simple method is to use the pytorch docker image, plain vanilla jupyter and our slightly modified notebook which you download and then upload into your instance the core of this guide will detail this method there is a custom docker image (fork) made specifically to run dd in docker jinaai/discoart https //github com/jina ai/discoart discoart can spin up somewhat faster and has a number of advanced features beyond the original notebook directions for using discoart on vast are here https //docs vast ai/disco diffusion we have created a video guide that shows all the steps for using disco diffusion on vast https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 https //www youtube com/watch?v=s4wifcrkez4 pytorch image + jupyter (recommended) 1\) select the docker image & config options open up the vast console and click on the edit image & config blue button select the pytorch pytorch image click again to open up the config menu then select the jupyter python notebook option we also recommend checking the box for jupyter direct https, although it is not required to explain the tradeoff direct https limits your search to machines with open ports which can be more expensive https direct also requires you to install our tls certificate or click through an "unsafe" warning on your browser the big benefit is that you get faster download speeds and a more reliable connection leaving the box unchecked is fine, you will then connect to jupyter through one of our proxy servers it isn't as fast, but doesn't require any further setup select 2\) allocate more disk space next you will want to allocate enough disk space under the edit image & config button there is a slider the default of 10gb can sometimes not be enough move the slider up to 15gb or more, especially if you are making videos 3\) select an offer here is the fun part! you can now select the gpu you want to rent for disco diffusion, leave the type as "on demand" as you don't want to be interrupted while generating art the offers presented to you are limited to machines that can support your image configuration options further use the filters to pick a 1x gpu (disco diffusion doesn't support more than 1 gpu) and the gpu type rtx 3090 is a great option rtx a6000 has more gpu ram which is great for larger models but is more expensive other gpus are also available hover over the rent button to get a breakdown of pricing once ready, hit rent to generate an instance the instance will now appear in the instances tab and will initialize 4\) download the modified dd notebook as a ipynb file the disco diffusion notebooks were created for colab we made a few slight modifications (to install a few required libs) so they will run on vast open the latest modified disco diffusion notebook here (5 6) and then file >download >download as ipynb you then have the ipynb file on your hard drive you can close the colab tab here are all our modified notebooks ( 5 6 https //colab research google com/github/jsbcannell/misc/blob/master/disco diffusion v5 6 vastai ipynb , 5 4 https //colab research google com/github/jsbcannell/misc/blob/master/disco diffusion v5 4 vastai ipynb , 5 2 https //colab research google com/drive/13zkuzdqolbfbvdcew8eyfnz mrnvpbbp?usp=sharing#scrollto=installlibs ) 5\) open the jupyter instance navigate to the instances in your account you should see your instance booting up once it is ready, click on the open button to open the jupyter interface note that it can sometimes take an additional 30 60 seconds for jupyter to start after the button appears if you get an error, wait a while and then reload spaces mgwtdaam0bo2skpvyo6q uploads biatmxba96wrwp8riyi9 rdy juypter instance ⚠️ warning if you selected the https direct method you will get a browser warning install the vast cert https //docs vast ai/instances/jupyter to get rid of that or just click through it 6\) run the modified dd notebook once you can connect to the instance, you will see the jupyter logo and a few files click on the upload button in the upper right and upload the ipynb file from 4 once that uploads, click on the notebook to open it if you are used to google colab, the interface for vanilla jupyter is similar with some differences one thing you will notice right away is that there are no clean input boxes/check boxes for modifying the settings scroll through the notebook and change the settings in the code directly review section 2 diffusion and clip model settings to see if there are any clip settings you want to change getting into the specifics of matching clip settings to your gpu (and gpu ram) is beyond the scope of this guide join the disco diffusion discord for help change the prompts at the end of section 3 doing a quick ctrl+f and searching for 'a beautiful painting' is a quick way to find the prompt settings modify the prompt to whatever art you want to generate now it's time to disco! once you have customized your settings, select cell >run all to start the code will start to execute from top to bottom the first couple of cells will install libs and download the clip models this can take 5 15 minutes the output of the code will appear in a text box spaces mgwtdaam0bo2skpvyo6q uploads wmy8fsvuhwzdw8avsmnb dd loading models scroll down to the bottom of the notebook to the end of step 4 right above step 5 you should see the in progress image spaces mgwtdaam0bo2skpvyo6q uploads 9t2xnymyqqygntuvkrzg disco time 7\) changing settings and downloading images after each image generates, it is placed into the images out/timetodisco folder you can then review your artistic creations they are easy to download individually after a while you will want to change the prompt or settings when the notebook is running, any changes that you make to it will not save for example, if you change the prompt, the notebook will not start working off that prompt after each setting change, you will need to hit the >> button to restart the kernal and then to run all the cells again the second time through it will not need to download any models and so should be fast zipping up all your images if you end up generating tens or hundreds! of images, there is a way to zip them up into a single file and then download them all to more conveniently download folders or a number of files, you can use the command line zip tool first open a new terminal by clicking new >terminal spaces mgwtdaam0bo2skpvyo6q uploads nggbmr0i6l36rzek1izx image then in the terminal copy and paste the following apt get install y zip hit enter to run those commands which will install the tool then to zip all of the files in the default images out/timetodisco directory zip all images zip images out/timetodisco/ note do no use spaces in your folder names, they cause headaches on linux! use the ' ' underscore instead additional help for discussion/help/advice running dd on vast find us on our discord, and make sure to check out the main dd discord custom docker image discoart instead of the pytorch image, you can use the custom jinaai/discoart docker image discoart can spin up somewhat faster and has a number of advanced features beyond the original notebook to use it, the steps are similar but instead of picking the pytorch pytorch image you select a blank template slot and then paste in the following docker image name jinaai/discoart\ latest also in the second line, use the following option to set the env variable e jupyter dir=/ jinai follow steps 2) and 3) to allocate more disk space and select an offer discoart has the notebook installed already in the docker image so once jupyter starts, hit the connect button to connect to jupyter install our tls cert (if you haven't already) for the direct https connection or else you will get a browser warning open the discoart folder and then open the discoart notebook change the prompt and any settings you want to modify then select cell >run all to start the notebook the results should start to appear quickly because the custom docker image already has downloaded the models examples using vast ai machines spaces mgwtdaam0bo2skpvyo6q uploads sacdwzfaycnharkav4ur dd cliff city 1 spaces mgwtdaam0bo2skpvyo6q uploads uvlxehktlsqr8ibbqo9g dd sorceress darksun t4 ​