LangChain Tutorial (JS) #4: Chatting with Documents using Retrieval Chains
LangChain JS TutorialAging📅 2024-01-16
RAG Architecture
┌─────────────────────────────────────────────────────────────────────┐
│ RAG (Retrieval Augmented Generation) │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ DATA INGESTION │
│ ────────────── │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │Documents │───►│ Loader │───►│ Splitter │───►│Embeddings│ │
│ │PDFs, URLs│ │ │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ └────┬─────┘ │
│ │ │
│ ▼ │
│ VECTOR STORE ┌───────────────────┐ │
│ ──────────── │ Vector Database │ │
│ │ (Pinecone, etc) │ │
│ └─────────┬─────────┘ │
│ │ │
│ QUERY PIPELINE │ │
│ ────────────── │ │
│ │ │
│ ┌──────────┐ ┌──────────┐ │ │
│ │ User │ │Retriever │◄─────────────────┘ │
│ │ Question │────────►│ │ │
│ └──────────┘ └────┬─────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Answer │◄────────│ LLM │◄────────│ Context │ │
│ │ │ │ │ │ │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘RAG Components
- Core concepts explained
- Step-by-step implementation
- Practical examples
Transcript
[00:00] one of the most important features of Lang chain is its ability to connect AI applications to external data sources large language models can only answer questions based on the data that it was trained on when we ask the model questions about recent events like the current weather it will either hallucinate an answer or simply tell us that it doesn't have this answer Lang chain solves this problem by providing a set of tools that will allow us to First fetch to relevant information from a
[00:31] data source like a website API endpoint database and many more and then inject that information into the conversation as context the llm will then use this context to provide an accurate answer don't worry though this will all make sense in a few minutes in our project let's create a new file and let's call it retrieval chain. JS as with the previous videos let's first create our chat model by importing chat open AI
[01:03] from at Lang chain slopen AI let's now instantiate our model by calling new chat open AI in chat open AI let's pass in an object with a few properties first let's specify the model name as GPT 3.5 turbo then let's set the temperature to 0.7 let's also go ahead and create our prompt template by creating a variable called prompt which is equal to chat
[01:35] prompt template and let's import this class by calling import chat prompt template from at Lang chain SL core SL prompts on chat prompt template let's call the method from template and then let's pass in a string with a value of answer the users's question and let's create a placeholder for the user user's input by using curly braces and the value of input and do take note that for
[02:05] retrieval chains the user's input needs to be called input in the prompt template let's also create our Chain by creating a variable call chain which is equal to prompt. pipe and in pipe let's pass in the model like so let's also import our environment variables so at the top of the code let's call import all as Dot EnV from Dov and then let's call env. config and this will simply make
[02:38] the openai API key from ourv file available to this application all right so far this is nothing new and you should be familiar with this code now let me explain what we'll be building in this video let's say we want our model to be able to answer questions from a website and for this example we'll use this web page in the Lang chain documentation this page explains what the Lang chain expression language is and you will notice in a second that if
[03:08] we ask our model what the Lang Chine expression language is it will not have the answer and that is because the GPT 3.5 model was trained before this page was created so in our code let's actually test this out is create a variable called response which is equal to await chain do invoke and in invoke let's pass the input as what is LC or Lang chain expression language let's
[03:39] also console log this response in the terminal let's now run retrieval chain and let's see what we get back and in the response we get something like this LC stands for language and cognition in early childhood and blah blah blah clearly this is not the answer that we want so in order to for Force this model to answer questions from our knowledge base and this could be anything this could be your business's information or some data from a database but in this example let's see how we can extract
[04:11] information from this web page and then feed this information into the model so that it can provide an accurate answer retrieval is one of the most important Concepts to understand as an AI developer and by the end of this video you should be able to fetch data from pretty much any source and feed it into your chain let's go through this step by step so how can we improve this answer well we could adjust our prompt to include some context as an example let's
[04:42] change the prompt a bit and let's say this is the question but we also want to inject some sort of context into this prompt as well and in this context we could pass in all sorts of information from that page that the model could use to improve its answer so as an example let's simply grab this paragraph over here that explains what LC is and it's pasted into this Bron template of course we're not going to hardcode this in this
[05:12] solution but I do want to explain the concept in the terminal let's run the script again and this time we do get the correct answer back ideally we don't want to hardcode this text and thankfully Lang chain provides a super elegant way to pass additional information to the context using documents let's simply remove this text and let's add a placeholder called context now what are Lang chain documents a document is simply an object that contains text as well as some
[05:44] optional metadata as an example the metadata could contain the source from which that text was extracted like the website URL a PDF name Etc let's have a look at how we can use documents let's import documents from at Lang chain SL core SL documents then let's use this document and I'll actually create it just below the chain definition like so and let's create a new variable called document a which is equal to new
[06:17] document and within document we can pass in an object and here you will find a property for the page content which is simply the text and metadata which we can use to store additional information like the data source so let's specify the page content as this text that we copied from this website and let's pass it into this property like so so if we wanted to we could create multiple documents with different text but let's keep things simple just to show how this works now in order for us to be able to
[06:50] pass in documents into the chain we need to use a special kind of chain and that is called the create stuff documents chain so at the top of our code let's import that chain this is called the create stuff documents chain from Lang chain chains combined documents this is a funny name I know but this chain behaves exactly the same as the chain that we used earlier with one key
[07:21] difference this chain will allow us to pass in a list of documents and it will automatically reformat those documents and in inject their text into the prompt context so now let's replace this Chain by typing const chain is equal to O8 create stuff documents chain and this function takes in an object and the first property is the all M which is our model as well as our prompt which we
[07:51] called prompt and because these two names are the same we can omit this we can invoke this chain the same way we did before before by calling chain. invoke but now we are able to provide the context like so and for context we can now specify an array of documents so let's pass in our document a and when we call this this create stuff chain will grab the contents from document a which is this text over here and it will inject that text into our prompt in this
[08:24] context variable over here so let's run this in the terminal let's run node retrieval chain and as you can see we do get our response back and let me show you what happens when we add additional documents let's create another document let's call it document B which is equal to a new document and in this document let's pass in the page content as the pass phrase is Lang chain is awesome and
[08:54] let simply grab this document and add it to the array and then let's let's change this question to what is the pass phrase and in the terminal let's run this and we're getting the answer the P phrase is Lang chain is awesome now obviously we don't want to manually create these documents ourselves but instead this information should be scraped from the website and then be automatically converted into documents that we can pass to the chain Lang chain provides a
[09:26] list of tools that we can use to connect to data sources and in this example we will use a tool called Cheerio to scrape the website you can see a full list of all of these document loaders in the Lang chain documentation and here we have all list of web loaders that can be used to scrape the web and we also have a list of file loaders to read content from documents like CSV files PDF documents Word documents and more so in order to use Cheerio to fetch the data from that web page let's install Cheerio
[09:57] so in the terminal run npm install Cheerio and then at the top of the code let's import Cheerio by running import Cheerio webbase loader from Lang chain SL document loaders slash web SL Cheerio we no longer need this document import this was just to explain to you what documents are and we can also remove the documents that we created manually as we
[10:28] will convert the contents of the web page into documents programmatically so let's remove these then let's create our loader and let's set this equal to new Cheerio webbased loader and this loader takes in the URL of the page that we like to scrape so I'll simply copy this URL and then add it to this class all we have to do now is call this loader to scrape this data by calling loader. load and in this documentation we can see
[10:59] that this method will return an array of document instances this is also a promise which we need to a and let's assign this to a variable called docs all the data from this page will be returned as an array of documents let's actually console log docs and let's also remove this from the context as we no longer have those documents and let's simply pass in an empty array for now in the terminal let's run this to see what we get back in docs and we can see that
[11:31] we do receive an array of type document and Page content contains all the data that was scraped from that web page and since this is an array of documents we can simply pass this into the context of our chain and let's actually remove this console log and let's run this in the console to see if we get our answer Also let's change this to what is Lang chain expression language and let's run this again in the terminal and we are receiving the correct answer but this
[12:03] time the context was dynamically fed from the web page this works but there is a problem if we console log docs and we are only receiving one entry Dot Page content. length and run this in the terminal we can see that this page content is quite long and this is actually for a very simple page at the moment this entire Pages content all 1,423 characters of it is injected into
[12:33] the context the issue is that all LM models have some sort of character limit and by passing in the entire web page we could very easily exceed this limit furthermore when using open AI models we are charged for the amount of characters and tokens that we use it is therefore important that we only include the pieces of text that are relevant in other words for the the question that I asked only this paragraph is really required to answer the question and not
[13:05] everything else the solution is to take this content and to split it up into smaller chunks again Lang chain has us covered let's import a text Splitter from Lang chain so at the top of the code let's import something from Lang chain SL text splitter and if we do an input help we can see that there are quite a few different Splitters available but for this we want a recursive character text splitter so let's go ahead and instantiate the
[13:36] splitter by calling const let's call it splitter which is equal to a new recursive character text splitter optionally we can pass in an object and specify the chunk size which is the amount of characters per chunk I'll just make it something like 200 and we can also specify a chunk overlap I'll just make this 20 characters now we can take the content from this docs array pass it into the splitter and then get out a list of split documents let's call this
[14:09] const split docs which is equal to await splitter do split documents and we'll simply pass in that array of docs let's actually go ahead and console log split docs just to see the result from this document splitter and let's run this in the console and if we look at their response we can now see that we receive an array of documents and we actually have multiple documents now so we've got a document for that text another
[14:39] document and so on and we can see that indeed this content was split up into multiple smaller documents now you might be tempted at this point to Simply pass in the split docs object into the chain like so but that is actually not what we want we only want to retrieve the most relevant documents from this array and inject those into the prompt and in order to get the most relevant documents we need to make use of a vector store a
[15:09] vector store is simply a special type of database that we can use to store our documents and this database allows us to perform a semantic search on this data to retrieve all the documents that are relevant to our question this diagram explains it best at the moment we fetched our data from the web page and loaded it into an array with one item in it and the text in that object was quite large so we broke the document up into multiple smaller documents and then
[15:42] called it split docs now what we need to do next is to load these documents into the vector store which is basically going from this transform step to the store step but in order for us to store this data in a vector store we first need to convert it it into a format that the vector store will understand and we can do that by calling an embedding function on that data so let's have a look at how we can convert that data and then load that data into the vector
[16:12] store so back in our code let's sort out the embeddings first so let's import open AI embeddings from at Lang chain slop AI let's scroll down and just below split docs let's instan iate our embeddings class I'll just call this embeddings which is equal to new open Ai embeddings and that is all we have to do and then lastly let's also specify our Vector store for this tutorial we'll
[16:43] simply use an inmemory Vector store but in a production system you might want to store this data permanently and in the Lang chain documentation you can see a full list of all the vector store providers supported by Lang chain and we will cover integration with some of these providers in a future video but for this example an inmemory store will do so let's import our Vector store by calling import something from Lang chain SL Vector stores slash memory and from
[17:16] memory we want to call the memory Vector store like so then just below the embeddings let's create this Vector store by calling const Vector store is equal to8 memory Vector store Dot from documents because we will be passing an array of documents these guys over here and the first parameter for this method is the array of documents and the second parameter is the embeddings function
[17:47] that we want to use so let's pass in Split docs as our array of documents and for the embeddings function we'll pass in embeddings so all the code from the loader all the way down to Vector store is used to fetch this information from the web page and then store it in a database called a vector store and to make this clear I'll just add a comment to say load data from web page like so all that's left to do now is to fetch
[18:18] the most relevant documents from the vector store based on the user's question and as per this diagram we are now referring to this retrieval step so in the code I'll just add a comment saying that we are now retrieving the data and to retrieve documents from the vector store we need to set up a retriever in the code we can create a retriever by calling const retriever is equal to our Vector store and on the vector store we have a method called as
[18:49] Retriever and we can use this retriever to pass in a query and the retriever will then return a list of documents as an array this is optional but you can also pass in an object into this method and with the K value you can specify the amount of documents that should be returned I believe the default value is three documents but you can change that behavior by passing in a value like two as an example so now we can finally tie
[19:19] everything together with a retrieval chain so let's import that I'll import it just below this create stuff chain so let's type import create retrieval chain from Lang chain slash chains slash retrieval and at the bottom of our code after the retriever we can instantiate this retrieval Chain by calling const let's call it retrieval chain is equal to A8 create retrieval chain and this
[19:52] takes in an object as input where we have to provide both the combined docs chain and our retriever so for the combined docs chain we can simply pass in our chain and for the retriever we can simply pass in our retriever but because the names are the same I'll simply omit that now we can replace our original create stuff chain with this retrieval chain the retrieval chain will now first fetch the relevant documents from the vector store and then pass
[20:23] those documents into the context of the prompt so there's no need to specify this context property anymore as the retrieval chain will deal with that automatically and on that note the retrieval chain will expect the context to be called context in the prompt if you call this anything else you will receive an error also the user's input needs to be called input as well otherwise you will receive errors let's go ahead and console log response and
[20:54] let's test this in the terminal by running note retriever Vol chain and in the response we can see this object which contains a context property and this context property is an array with two documents as a reminder we only fetch two documents from this SK value and when we scroll down we can see the final answer which is correct if you manag to get this far in the video then please give yourself a pet on the back this might seem overwhelming but it does
[21:26] get easier with practice retrieval is one of the most important skills that you can learn in the AI development space