Spring AI is a new addition in the spring framework which helps in developing java applications in AI domain. It helps you in wiring up different LLM models and vector databases. It follows the same convention as other spring modules where you work with configuring the external entities using configuration files and framework will help you in providing the required beans for your development.
Setup
The setup comprises of local installation of Ollama (using installer) and pgvector (using docker image). Ollama helps us in running LLM models locally, we will be using llama2 as a chat model and nomic-embed-text as embedding model. You will be amazed to see how easy it is to run LLM models locally using Ollama.
From the application end, we will be using corresponding spring starters for ollama and pgvector database. For UI, we will be using Vaadin. It will be a great experience to see how easy it is to build a neat UI with the comfort of java, without getting into the hassle of HTML and JavaScript.
Application Flow
Below are the functionalities which will be part of the application.
Upload Data
User will upload text data using the UI. Application will interact with embedding model (nomic-embed-text) to get the embeddings of the text, and then eventually store them in the vector database (pgvector).
Search Data
User will search for the text in the uploaded data. Application will interact with embedding model (nomic-embed-text) to get the embeddings of the search query, and then eventually perform a similarity search in the vector database (pgvector). The result of the search is displayed back to the user.
Summarize Data
User can perform the summarization of the search result. Application will interact with chat model (llama2) to get the summary of the search result using a simple prompt.
