Ollama + Webui

ollama & webui documentation below is a step by step guide on how to configure and run ollama our template will automatically setup open webui as a web based interface as well as expose a port for the ollama api r1 (deepseek r1 70b) is used as the example model in this guide ollama has many r1 models available to use which the webui can download the larger the model, the more total gpu ram and disk space you will need to allocate when renting your gpu the models page has a drop down menu showing the model name and total gpu ram needed to run it you will also need at least that much disk space on the instance find and rent your gpu setup your vast account and add credit review the quickstart guide to get familar with the service if you do not have an account with credits loaded select the ollama template click on temp lates and select the recomended ollama template open webui (ollama) click on the play icon to select the template you will then go to the search menu to find a gpu click on the readme link at any time for a detailed guide on how to use the template disk space from the search menu, ensure you have sufficient disk space for the model(s) you plan to run the disk slider is located under the template icon on the left hand column large models (e g , 70b parameters) can require dozens of gigabytes of storage for deep seek r1 70b, make sure to allocate 50gb of disk space using the slider vram requirements check that your gpu vram is sufficient for the model larger models require more vram for deep seek r1 70b, we will need at least 43gb of vram find the slider titled gpu total ram and slide it ot 44gb example r1 (deepseek r1 70b) we recomend a 2x 4090 instance with 50gb of disk space steps to open the webui with ollama after the instance loads, click the "open" button this will initiate the instance portal with links to all the services running on the instance click the "open webui" link create an admin account upon first use (or if prompted), create an admin username and password to secure your instance you can add additional users in the admin panel model download click on the admin panel > settings click on the models tab click the download icon to manage models put in the model name to pull directly from ollama com for our example that would be deepseek r1 70b wait for the model to fully download start a new chat once the download is complete, return to the webui main page and start a new chat session you can now test the model by sending prompts enjoy! ollama api usage ollama provides a direct api that you can call outside the webui by default, it is available at https //instance ip\ port 11434 authentication token when making requests, you must include an authorization header with the token value of open button token this token is typically displayed or stored in the webui settings or environment variable sample curl command curl k https //instance ip\ external port/v1/completions \\ h "content type application/json" \\ h "authorization bearer open button token" \\ d '{ "model" "deepseek r1 70b", "prompt" "san francisco is a", "max tokens" 128, "temperature" 0 6 }' k allows curl to perform insecure ssl connections and transfers as vast ai uses a self signed certificate replace instance ip and external port with the externally mapped port for 11434 from the ip button on the instance update the authorization header value to match your open button token you can get that from any of the links in the instance portal or from the open button on the instance card modify the prompt, model, and other fields (max tokens, temperature, etc ) as needed