LangChain Tutorial (Python) #5: Adding Chat History to Chatbot
LangChain Python TutorialAging📅 2024-01-26
Integration Flow
┌─────────────────────────────────────────────────────────────────────┐
│ INTEGRATION ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ USER REQUEST │
│ │ │
│ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ ────► │ │ ────► │ │ │
│ │ Service A │ │ AI Engine │ │ Output │ │
│ │ │ ◄──── │ │ ◄──── │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │ │ │
│ │ │ │ │
│ └────────────────────────┴───────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────┐ │
│ │ RESULT │ │
│ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Integration Points
- Core concepts explained
- Step-by-step implementation
- Practical examples
Transcript
[00:00] In the previous video, we created this retrieval chain that is able to fetch information from a webpage, and we are then able to ask questions about the content of that webpage by calling this invoke statement. In this video, we will improve this retrieval chain and turn it into a conversational chatbot. This means that the model will have access to the conversation history so that we can ask follow-up questions. I think you're really going to enjoy this video, and by the end of this video, you
[00:30] will have your very own conversational chatbot, which is also connected to external data sources. Let's start by copying the code from the previous video. You can find a link to the source code in the description of this video, and in our project, let's create a new file, and let's call it conversation retrieval dot by, and let's paste in that code. We'll leave most of this code the same, but I do want to make a few small changes. We can improve the quality of the responses by increasing this chunk size to 400 characters,
[01:03] and let's change the k value of the retriever to 3 documents. And lastly, let's change this from context to answer, and let's test this in the terminal to make sure that this code still works by running python conversation retrieval dot by, and we are passing in the question, what is lc el? And as with the previous video, everything is still working. Great. The first change I want to make is for our code to fetch the question from the
[01:34] terminal instead of us hardcoding it like we're doing at the moment. So let's see how we can implement this. First, let's move the code that calls our chain into its very own function. I'm therefore going to cut this code and let's create a new function. Let's call it process chat and let's add that code to this function like so. Then instead of printing the response, let's rather return the response like so. Then let's also add if name equals main,
[02:05] we will execute all of this code like so. This is just a python convention and will ensure that this code only executes if this is the main file that's being processed and not if it's imported by another file. So when we run the file, we will extract our documents from this web page. We will then create a vector store using those documents. And then finally, we will create our retrieval chain. We can then pass chain into process chat. So let's actually add this to the parameters of process chat. And then in this code, we can simply call
[02:37] process chat and pass in the chain like so. So another value that we would like to make dynamic is this question. So let's replace this hard coded text with question and let's also pass in question as a parameter. So on process chat, we can simply pass in our question as a parameter, but let's fetch that value from the terminal instead. Thankfully, that's easy to do in python as python provides a built in function to fetch values from the terminal. Let's create a variable called user input
[03:09] and let's say that equal to input and let's add some text to this like you. And then let's grab this variable and let's pass this into the process chat function. It's also assigned this to a variable called response. And lastly, let's print the value of response and I'll prefix it with assistant. Let's go ahead and run this in the terminal by running py conversation retrieval. We can now enter our question. Let's do something like what is lc el
[03:40] just to make sure that this still works. Cool. And now we are getting our response back so we can improve this by adding this code to a loop. So just above user input, let's say while true and this will put us into an infinite loop. So let's also give the user some way of exiting out of this by adding if user input is forced us to lowercase equals exit. Then we can simply break out of this loop. Now let's give this a spin by running this in the terminal again. So let's start with hello
[04:12] and we get a response back. Let's ask it a question like what is lc el streaming and we get our response back. Let's say my name is Leon and we get this message. Hello Leon. How can I assist you today? Now let's try this. What is my name and you would expect the model to be able to tell me what my name is. But unfortunately it can't tell me what my name is because at the moment our chain does not have a conversation history. Let's exit out of this.
[04:43] And let's have a look at how we can add conversation history to this retrieval chain. Let's create a variable for our chat history and I'll add it right before this while statement. I will call it chat history which is equal to an empty list. So basically how this is going to work is we will keep appending the user input as well as the assistance response to this list. But the entries in this list needs to follow a very specific schema. And there is a schema that represents the human message and a separate schema that
[05:15] represents the AI message. Let's have a look at those schemas. So we can import those schemas from Lang chain. So at the top of our code let's say from lang chain core dot messages import the human message and the AI message. So let's go back to our chat history and let's manually add a few entries just to see how this works. First let's add a human message and human message takes in a content property and let's enter something like hello.
[05:46] Then let's add an AI message with the content. Hello. How can I assist you? Then let's add one more human message with the content. My name is Leon. Now we just need some way of passing this chat history into our chain. So first let's pass the chat history into this process chat function. Then let's also scroll up to the process chat function and let's add chat history as a parameter. And what we can do then is pass the chat history into the chain like so.
[06:17] And then let's also extend our prompt template to include the chat history. So you might be tempted to simply add chat history to this prompt template. But there's actually a way more elegant way of doing this. First let's remove this from templates method. Let's replace it with from messages. And this is honestly my preferred way of building up these prompt templates. This takes in a list of tuples and for the first value we can pass in a system message. And for the system message let's enter
[06:49] answer the user's questions based on the following context. And we then provide our context variable just like we did before. Let's also pass in our human message. And for this we will simply pass in a placeholder for our input. Now in order to add the chat history we will make use of a placeholder object. And we will add it just before this human message. And this message placeholder object will allow us to pass in a list with these message schemas.
[07:20] And it will automatically type the content from these objects and build up a string value. So at the top of the code let's import that message placeholder object from langchain core dot prompts import the messages placeholder. And now let's add that to our prompt template. So just above the human message let's call messages placeholder. And this takes in a property called variable name which we called chat history. Let's test this in the terminal and let's
[07:51] ask it a question based on this chat history. So let's simply ask it what is my name. And as you can see it was able to remember my name. Right so let's exit out of this and let's dynamically populate this chat history. Let's remove this sample messages so that we start off with a blank chat history. Then just below the response let's enter chat history dot append. And let's append a human message with the content of user input.
[08:22] Then let's also add the assistant response by typing chat history dot append a message with a content value of response. And this will now dynamically build up this chat history list. So let's test this. Let's say what is my name. Right so it does not know my name. Let's now say my name is Leon. Great and now let's ask it what is my name. And this time it was able to recall my name.
[08:52] So let's have a look at how we can improve this code even further. You might recall that we have this retrieval chain over here which combines our chain with a retriever. So each time we interact with this chain the user input is passed to our vector store. And this will then fetch the most relevant documents based on the input from the user. And I think the best way to demonstrate this is by using a tool called Langsmith. And I highly recommend that you sign up for the waiting list to
[09:23] get access to Langsmith. But in a nutshell Langsmith makes it very easy to debug and troubleshoot our Langchain chains. So let's click on the latest run and that's the one that we just did now. And we can see the input coming in as what is my name. And our code then calls this retriever and it's passing in what is my name to the vector store. And it then returns the three most relevant documents. And at the moment the retriever is simply passing the latest question from the user
[09:54] to the vector store which is not ideal. Ideally we want to generate a query based on the conversation history so that we can fine-tune the results that we get back. So the way we will solve this is to create a history aware retriever. And this special retriever will receive our chat history and then generate a unique query based on the latest input from the user and the conversation history. Let's have a look at this in our code. So this retriever is simply taking the
[10:26] latest input from the user and passing it to the vector store. But Langchain offers an additional function that we can call to attach chat history to this retriever. Let's import that function from Langchain. So at the top of our code let's say from langchain.chains.historyawareretriever import create history aware retriever. Just below retriever let's create a new variable. Let's call it history aware retriever is
[10:58] equal to create history aware retriever. And this takes in an LLMS input which we can set equal to model. And it also takes in a retriever which we can set equal to our retriever. And the reason this needs an LLM is because it will use an LLM to generate this new query by combining the chat history with the latest input from the user. And since this is using an LLM we also need to provide a prompt. Let's set prompt equal to and we will
[11:30] create a new prompt for this. I'll just go ahead and call it retriever prompt for now. And let's actually go ahead and create that prompt. So let's create retriever prompt. Let's set that equal to a chat prompt template dot from messages. And let me explain how this is going to be structured. This prompt will be used to combine the chat history with the latest user input. And it is responsible for generating a unique query for the vector store. So first let's pause in the chat history
[12:02] by calling messages placeholder again. And the variable name will be called chat history. Then secondly let's pause in our human input by calling human. And we can simply add a placeholder called input. And now we just have to specify some instructions or system prompt. But I'll just add it to a human field. And let's say given the above conversation. So this is now including the chat history and the human input generate a search query to look up in order to get information
[12:34] relevant to the conversation. And you will see in Langsmith what his final result is. So now we can simply take this history aware retriever and then replace retriever from the retrieval chain with this history aware retriever. Like so. And that is actually all we have to do to fix up our retrieval chain. Let's test this again in the terminal. And let's say something like my name is Leon again. And let's also ask what is LCEL. And let's have a look at it in Langsmith.
[13:06] Let's refresh this. Then let's open up the first entry. And now we can see within this retrieval chain that we now have a call to chat open AI. And we can also see our chat history is reflecting in here as well as that system message. And in the output we now get that query. And if we have a look at the retriever it is that generated query that is now being passed to the vector store. And hopefully you can see the benefit of doing this. As we go through this conversation
[13:37] history this query will be fine tuned. It's also add something like what is LCEL streaming. And hopefully the emphasis on my name will become less and less important. And the focus of the query will now be on LCEL and streaming. Let's go back to Langsmith. Let's refresh this. Let's open up this last call. Let's have a look at this retrieval document chat open AI step. And we can now see that the query is way more fine tuned.
[14:08] Which will also produce way better results in the retriever step. Congratulations. You now have a fully functional conversational chatbot that is connected to an external data source. If you like this video and the series please consider subscribing to my channel. And please hit the like button and share these videos.