Large Language Models (LLMs) have limits. They don't automatically know your data, can hallucinate facts, and struggle with very large datasets—forcing you to manage information manually.
Retrieval Augmented Generation (RAG) addresses this by supplying the LLM with your own data, no matter the size. Importantly, it is also able to do this in a scalable way that doesn't require us to endlessly scale our models. Here is a technical dive into how you can use Zero Point AI's RAG LLM to handle large datasets for better, data grounded results.
Step 1: Processing your Dataset
Upload any text-based files. These will be what the AI refers to when generating its answer. Think of it like a references set or bibliography for a scientific report, where all of the information cited in the AI's answer comes from this list.
Step 2: Chunking
Behind the scenes, your dataset is "chunked" into smaller sections of approximately 175 words – on the order of a paragraph.
Step 3: Embedding
Specialised models are then used to turn the chunks into a vector in high dimensional space, reflecting the meaning of the chunk using upwards of 768 datapoints. This makes the chunks comparable, statistically or geometrically. These are stored in an easily searchable database, so they can be referred to later by the AI – but only if it is relevant.
Step 4: Question Processing
When you ask a question, it is also embedded, into a long vector which can be compared to the context. Only the vectors that are most similar to question are pulled from the database and passed forwards, making RAG LLMs much more efficient and accurate compared to non-RAG LLMs.
In reality, there is some processing to your question to be done first to make it more comparable to chunks. In Zero Point AI's system, this takes an interim LLM which elaborates, word associates and paraphrases your question to get the most accurate results.
Step 5: LLM Generation
The context has now been assembled – sometimes upwards of 4000 words, depending on the question and the exact setup chosen for the RAG LLM. This can then be passed into the Large Language Model, along with your original question for answering.
A mark of quality for RAG LLMs is their additional ability to refer back to the context documents, enabling a human check of the accuracy of the AI. Zero Point AI's RAG LLM uses referencing back to the precise chunk that was used by the LLM to generate that part of its answer. That isn't typically the case for normal LLMs.
RAG LLMs streamline work by handling large volumes of information, improving accuracy, and freeing teams from time-consuming, dull document review.