Is RAG a good solution for modern chatbots?

The Modern Internet

The modern internet is a real jungle. Every Google search, no matter how simple, brings us thousands of results in a fraction of a second. But have you ever asked yourself how a search engine like Google is able to bring us such accurate results, even though websites are constantly updating? The answer is simple - search engines perform crawling on every site that allows it, thereby ensuring that the information they display is (almost) always up-to-date.

RAG as a Search Tool

If like me, you also find yourself scrolling on X (formerly Twitter) every day, there's no doubt you've already encountered RAG, or its full name: Retrieval Augmented Generation. So how does RAG work, you ask? In industry terms: given some data, we run an Embedding model that converts our information into a vector, and when we receive a Prompt, we can calculate Similarity between the information stored with us and the Prompt. In human terms: we store all our information as a long list of numbers. When someone asks a question, we can bring the most relevant information from all the information we stored according to a metric that checks how similar that information is to the question we received. This way, we can pass to our model the information we received through RAG, and it can provide an answer based on that information!

The Problem with RAG

So on the face of it, RAG sounds like an excellent solution! I receive a question, bring the relevant information, and through it my model generates an answer. Where does it fall short? When we look at the information stored with us, suddenly things don't look so rosy... When you enter a chatbot creation platform like Chatbase or Kappa and give them your website address, two things happen:

1. They run a Crawler that goes through your entire site and extracts all the information from it in a clean form suitable for language models.
2. All the extracted information goes through an Embedding process (a process that converts text to numbers) and is stored in a vector database.

When a user enters your chatbot and asks a question, a search is performed for the most relevant information from all the stored information, but what if my question relates to new information on the site? What if I asked about a product that didn't exist a few days ago? What if the site published a new blog last week? In these cases, RAG won't be able to bring this information because it doesn't exist in the database!

Will RAG Still Be Needed?

I believe that every algorithm has some use, and indeed RAG is no different. In cases where the information I want to store doesn't change, RAG can be a very good and efficient solution. Our computers are very good at arithmetic operations, so calculating Similarity and returning relevant information from a vector database are operations that can be performed very quickly. But, if my information changes and accuracy is important over speed, other solutions win.

KedAI's "Secret Recipe"

Without diving deep, because this is our secret - given a question, KedAI's chatbots don't necessarily perform RAG. Every chatbot built with KedAI's infrastructure is capable of learning about sites and updating in real-time, while the site changes. Our algorithms don't require any user intervention, and everything happens behind the scenes. Want to see the capabilities of our chatbots and examine the results yourself? We offer a free plan through which you have 50 free messages every month! You can use this plan to build yourself a chatbot that will successfully answer every question with the most up-to-date information you have on your site! Get started now at www.kedai.co.il.