Langflow + Ollama

langflow is a node based agent builder you can use from your web browser while it integrates with many frontier language models it also has a fantastic ollama integration which makes it really easy to use with open weight models as well as custom fine tunes we have two templates you can choose for this guide the langflow template provides both ollama and langflow installed within the instance you can also use the ollama standalone template to integrate with a local langflow installation via ssh local port forwarding the choice is yours for this guide we will use the langflow bundled template before moving on with the guide, setup your vast account and add credit review the quickstart guide to get familar with the service if you do not have an account with credits loaded initial setup let's get started with the configuration there is not much you need to change here but it's a good idea to create a customized version of the template so ollama automatically downloads your preferred model find the template you can find the langflow template in our recommended templates page before loading it up, click the pencil icon to open up the template editor langflow template card custom configuration in the template editor you'll find two really useful configuration variables ollama model is the most important variable here you can choose which model should be downloaded when the instance starts langflow args allows you to pass alternative startup arguments to the langflow application the defaults should be fine for this demo, but you are free to change these as you need when you have finished entering your settings click the ' create & use ' button to save your copy of the template you'll be taken to the search interface where you can choose an appropriate gpu instance to run your model you can access your custom template in future from the 'my templates' section of the templates page starting the instance it's now time to use your template to start a gpu instance choose a gpu the most important consideration when picking an instance to run laguage models is the vram for best performance, your model weights must fit into the gpu vram with room left over for the context window you do not have to use a single gpu when running llms sometimes a multi gpu setup can be as effective of better than a single high vram instance rent an instance when you have found a suitable instance it's time to click the ' rent ' button this will start the loading phase if you are not sure which instance to choose try one there is no minimum rental period and if it is not suitable you are able to destroy that instance and start another, paying only for the time the instance was in the 'running' state accessing the instance after a short time, your instance will be ready to access simply click the 'open' button to get started you will now find the instance portal has opened instance portal this page gives you easy access to both the langflow application and the ollama api click langflow's 'launch application' button it will take some time for langflow and ollama to be installed and for the ollama model to download you can monitor the loading status in the instance portal 'logs' tab getting started with langflow after opening langflow, click the ' create first flow ' button while langflow is extremely powerful, for this example we will create a simple blow post writer select content generation > blog writer initially, the flow will look like this blog writer flow we will need to replace the language model with the ollama alternative to make use of the gpu and avoid having to make api calls to external services click on the language model node and using the three dot icon, choose delete delete language model next, from the left side menu, select the ollama component and drag it to the space created by deleting the original language model component add ollama node now that is in place it must be configured to communicate with the ollama api enter http //localhost 11434 in the base url field you'll need to then select your ollama model and re connect the nodes as shown below ollama node connected if the model field does not immediately show your available models, simply toggle the 'tool mode enabled' switch configuring the workflow you could run this node immediately, but first let's make some minor modifications change the url in the url node to https //vast ai and set the depth to 2 change the text in the instructions node to use the references above for style to write a new blog/tutorial about how vast ai can empower people who want to leverage affordable gpu resources run the workflow simply click the playground button followed by the run flow button and wait for the agent to learn about the subject matter and write a blog post it'll only take a few seconds completed blog post advanced usage this short guide serves only as an introduction to langflow, but it is extremely capabale and easy to use with some practice we recommend that you check out the excellent documentation to assist you in creating complex projects remember, any language model component can be replaced with the ollama component, and any agent component can be configured to use ollama as a custom provider