Build a Chat Bot using Chat Models | JavaScript Tutorial #2
LangChainVintage📅 2023-05-11
Build Process
┌─────────────────────────────────────────────────────────────────────┐
│ BUILD CHATBOT WORKFLOW │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ PLANNING │
│ ──────── │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Define │───►│ Choose │───►│ Setup │ │
│ │ Scope │ │ Stack │ │ Project │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ DEVELOPMENT │
│ ─────────── │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Build │───►│ Add AI │───►│ Test & │ │
│ │ Features │ │ Logic │ │ Debug │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ DEPLOYMENT │
│ ────────── │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Deploy │───►│ Chatbot │ ◄── LIVE! │
│ │ to Cloud │ │ Running │ │
│ └─────────────┘ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Key Components
- Project architecture
- Core features implementation
- Best practices
Transcript
[00:00] hello and welcome back in the last video we had a look at setting up a line chain application in our node environment and we also had a look at the fundamentals of using Lang chain if you haven't watched that video yet I highly recommend watching that video first and then coming back to this video in this video we'll move away from standard LL aims and we'll have a look at chat models chat models are very similar to standard LL aims and they do use an llm under the hood but the big difference is that for llms we pass in a prompt and we
[00:32] get a response back and that's kind of the end of the conversation with chat models we no longer deal with simple text in and text out scenarios but instead we deal with full-blend conversations like a chat history so let's go ahead and create our first chat application in the root of this project I'll create a new file and I'll call it chat demo.js let's go ahead and create our model so at the top of the code we'll import chat open AI from Lang chain slash chat models slash open Ai and
[01:06] hopefully you can see a small change in the import here in the previous videos we imported from the llm package openai and the class that we imported was just open AI if I switch back to the demo file for a second this is what I'm referring to so in this example we are accessing the openai lrm directly but in this file we are now importing the chat open AI wrapper instead so after importing this we can instantiate our new model by piping const model equals
[01:37] and we'll create a new instance of the chat openai class this class takes in an object as input and we can also specify the temperature which I'll say it is zero so far this looks very similar to The Standard llm but there is one major difference when calling this model so this one call this model by typing model dot call and this is where the big difference comes in this chat model takes an array as input and it's in this array that we will be passing the conversation to the model and this
[02:08] conversation follows a very specific schema that we have to adhere to so it will not be possible to just pause in a string like this and expect it to work but instead we need to make use of the schemas provided by Lang chain so let's remove this text and let's import those schemas from line chain so at the top of the code we can import human chat message from Lang chain slash schema so we can now use this human chat message Clause to pause in our message to the model so in this array or pause in a new
[02:40] instance of the human chat message clause and this class takes in our message as input and let's say that for this example we want to create a chatbot that can translate text to a different language so let's ask the model to to translate this sentence from English to office Afrikaans which is my native language and the text that we want to translate is I love programming so whenever we want to pause a message that's generated by the user we will
[03:11] make use of this human chat message schema the scroll method returns a promise which we need to avoid and let's assign the result to a variable called raise there's also console log this result this model also expects the openai API key to be available so let's import our environment variables like so we can now go ahead and save this file right so let's go ahead and test this out so in the terminal we can run node and then the file name which is chat demo and let's press enter and after a
[03:43] second ik this response back with the text access which is Afrikaans for I love programming and the name is undefined which is fine and just a little side note and this is something I didn't mention in the previous video when instantiating our model we saw that we are able to pass several properties to the interface like the temperature streaming Etc there is one more property that you should be aware of and that is a property called Model name if we pull up the openai documentation for a second
[04:15] and if we go to models we can see all the available models in this list by default langchine is using gpt4 as the model but if you wanted to use something like GPT 3.5 which is actually a very cheap model compared to gpt4 we can simply copy the name of the model and then in our code we can just pause in the model name and if we test this out it should still work so let's run the script and there you go our model is now using GPT 3.5 turbo so I just wanted to
[04:46] mention that you can change your model if you wanted to so at the moment we are simply passing in our human chat message but there are other chat schemas available that you should be aware of so if we flip back to the openai website for a second this will visually show you what I'm referring to at the moment be passed in a message here for the user and that message was something like translate this sentence from English to Afrikaans or whatever your prompt was after running this we got this response back from the AI with a role of
[05:18] assistant so with this example we can see the different roles in action so we've got our user role which is the user chat message we've also got this message coming back from the model with a role of assistant and this is also known as the AI chat message then on the left hand side we can see the system prompt as well and this can be used to Prime the AI the behave in a certain way or to respond in a certain way and an example of a system message in this scenario would be something like this where we tell the AI that you are a
[05:50] helpful assistant that translates English to Afrikaans so this represents the three different chat schemas that is available in line chain with got the human chat message the AI chat message and then a system chat message if we flip back to our code for a second we can see this in action so let's say that along with this human chat message we also want to pause in a system chat message we can do that by also importing system chat message from the schema
[06:20] package we ideally want to send that system chat message before our human chat message so just above the human chat message I will pass in a new instance of the system chat message and I will provide it with that priming instruction that says you are a helpful assistant that translates English to Afrikaans and I'll just add a comma at the end of this so now we are passing in our system message as well as our user prompt and after running this in the terminal we are still getting our response back we can also get our model
[06:52] to generate multiple completions so at the moment this call is giving us one generation so what if we wanted to gen generate multiple completions within a single call as an example we want to translate this text to Afrikaans as well as Fringe so for multiple completions we can actually call the generate function on our model generate will provide multiple completions and with generate we actually provide an array of arrays let's actually remove this array so we
[07:23] are only left with this generate function call we will then pass in an array and I'll just paste in back our original prompt and what we can do now is provide another array as a value so I'll just copy all of this and paste it in here for the second item I'm going to change the system chat message slightly to say you are a helpful assistant that translates English to French this time and then in our human chat message or change Afrikaans to Fringe I'll go ahead
[07:54] and save this and then this is basically what the score looks like now we are now calling the generate function and passing in these two arrays as values let's run this in the terminal and in the terminal we got this response back and we can see that we are getting this object called Generations which seems to contain two objects so back in the code I'm just going to change our console log to write out response dot Generations let's execute this again and now we can see the two values that were returned so we are getting an array and the first
[08:25] translation is our Afrikaans translation followed by the French translation let's talk about prompt templates with chat models we can also use prompt templates with our chat models so that we can replace these texts like the system chat messages with prompt templates and then replace the input and output languages with variables this way we've got a very dynamic system let's have a look at how we can Implement prompt templates with our chat models so in order to
[08:55] demonstrate prompt templates I'm actually going to just copy out this code for now as well as the quality generate function and I'll remove this console log as well instead of calling these schemas directly I'm going to remove this import and what we'll do instead is at the top of the kite we'll import a system message prompt template from Lang chain slash prompts and we actually need two more Imports we need the human message prompt template so
[09:26] basically we will remove the text that we pass into the system chat message and the human chat message with these prompt templates and in order to tie these two messages together into a single prompt we will also import the chat prompt template we can Define our new prompt template in The Code by typing const translation prompt we'll see that equal to the chat prompt template and on this class there's a method called from prompt messages and what we're going to
[09:56] do is we're going to pause in the system message as well as is the human message so these two messages that we use down here we will pass that into this class along with the variables and that will generate a prompt string that we can pass into the final model or chain this function types in an array as input and this will be the array of messages so the first message that we want to pause to it is our system message so we'll pause system message prompt template on this class there's a method called from
[10:27] template and within this function we can pass in this string so I'll just copy it from this example and let's paste that into this function and now we can Leverage The Power of prompt templates we can now replace this from language and to language with variables this replies English with input language and will replace this language with output language right so this will now allow us to pass in variables when calling our prompt after this is the message we also want to pause in our human message so
[10:58] now we can call our human message plan template clause on this class is a method called from template and this is going to be the phrase that we want translated like I love programming but instead of passing the value like that we will rather pass in a variable and I'll call this text so this translation prompt object will now be able to generate a prompt for our model using the prompt templates that we passed into it we can have a look at the prompt that this object will generate by calling our
[11:30] translation prompt object on this object is a function called format prompt value and this function takes in an object and the properties of this object are the variables that we specified in our prompt string so let's pause in the input language as English or pass in the output language as Afrikaans and then for the text we'll pass in I love programming this function returns a problem numbers so we can just await
[12:01] that result and let's assign the result to a variable called formatted prompt it's console log formatted prompt to see the result so let's run this in the terminal and in the terminal this will look very familiar to you we now have an array of messages and in this array we've got the system chat message with a text you are a helpful assistant that translates English to Afrikaans and we have our human chat message with a text I love programming so it's taken our prompt templates and converted it into
[12:31] this format that we used earlier let's remove this console log let's add back in our model I'm going to go ahead and delete all of this code and let's now pause in this full method prompt to our model instead so we can call our model on the model we will now call the generate prompt function because we're using a prompt template this tags in an array as input and within this array we'll simply pass in our formatted prompt it's a weightless response and let's assign it to a variable called response let's go ahead and console log
[13:03] this so I'll type console.log dot raise and this will give us an object called Generations let's go ahead and run this in the terminal and after a second I get this response back so at the moment we are creating our formatted prompt and in staining that to the model directly but in Lang chain we want to do things slightly different instead we want to use a chain and it's a Chain's responsibility for merging our prompt with our model so we can also add chains to chat models by importing something from the link chain package at the top
[13:34] of the code we will import the llm chain from line chain slash chains I'm going to remove this call on the model as well as this console log then what we'll do is we'll Define a new chain which is equal to new llm chain this takes in an object with a property for prompt and for the prompt we will pass in our four method prompt it also takes in the LR LM which was set equal to our model and now instead of calling the model directly we can call our chain on the Chain this is
[14:07] a function called call and we can now pause in the variable values into this object so we need to pass it and input language like English we'll give it the output language as well which was Afrikaans and we need to specify the text which is I love programming the score function retains a promise which we need to await and it's assign this to a variable called response we should now be able to console log this response so call console log response and let's run
[14:40] this in the terminal and apologies we do get an error message here and that is because we no longer need this so sorry for that we are now passing the values directly to the chain so it's this thing here this translation prompt this is what we need to pause to our llm chain so let's save this value and try again after running it in the terminal we can see this response coming back great we are now using chains to combine our model with our prompt templates next let's talk about tools and agents in the
[15:11] previous video we discussed what tools and agents are and for chat models it's exactly the same so let me show you how we can introduce agents and tools to our chat model let's go ahead and clean up this code for this example we won't be using the chain so I'm just going to remove that we do need our model and we're not going to use this translation prompt either so I'll delete that as well I'm also going to clean up these Imports so that we have a fresh start right so let's have a look at tools and agents so in this example we want to ask
[15:43] our chatbot how many people live in the US we know by now that GPT does not have up-to-date information so because GPT does not have direct access to the internet we will use an agent to determine that GPT needs assistance and it will assign a relevant tool to the model for fetching information from the internet so the first thing we need to do is to specify the list of tools that we can make available to our model so I'll Define a new variable called tools which is equal to an array of tools so
[16:16] as with the previous video we will use the serp API to allow our model to Google for results so we need to import serp API from Lang chain slash tools then in the list of tools we will pass in a new instance of serp API and serp API takes a few values as input the first is the API key which we can find in our environment variables by going to process.env dot serp API key the second
[16:47] value is an object with the location details if you want you can specify your physical location I'll just leave that blank for HL or set en and for GL all set U.S and this is the only tool that we want to make available to our model and then we need to create an agent we can do that by creating a variable called agent and for agent we need to import something from Lang chain so what I'll do is also import chat agent from Lang chain slash agents and then we can
[17:20] set agent equals to chat agent and on chat agent we've got a function called from llm and tools so what we're saying here is we want to create a new agent from one of our llms and the array of tools so in fact we actually don't need the model here so for this function we need to pass in the llm which will be a new instance of our chat openai model so alternatively what we could have done here is instead of calling it that way
[17:51] we could have moved this model up to above the agent and we could have passed in the model as input or you can Define it in line other way is completely fine let's actually leave it like this the second parameter is the list of tools which we defined as tools so now we have an agent that will use our openai model as well as the list of tools to perform their actions so the agent can't do much by itself and in order to use the agent we need to use an agent executor and the agent executor will effectively call
[18:22] this agent in a loop until the result is found so let's also import agent executor from the agents package so below agent we can now create a new variable for our executor we will set that equal to our agent executor and we'll create our executor from our agent and the available tools so on agent executor we will call the function from agent and tools this takes in an object and for this object we need to pass in the agent which we called Agent the
[18:53] second parameter is the list of tools which we also called tools so I know this can seem quite a one thing but basically what we're doing is we're creating a new agent and agents are quite intelligent in the sense that they will call your llm model the llm will give a response back indicating that it's unable to fetch that information based on its training data the agent will then use tools to enrich The Prompt that is passed back to the model and the agent can only use the tools that is assigned to it an Executor is
[19:24] responsible for calling the agents until they find a response and the executor can also have a list of tools assigned to it I hope that makes sense now that we have our agent and our executor we can now call the executor to start this process we can do that by calling executor dot run and we can now pass our prompt to the executor so let's say how many people live in the US in 2023 the Run function returns a promise which we
[19:56] need to go right and let's assign the response to a variable called response and let's console log this response and let's go ahead and run the script in the terminal and after a few seconds we got this response the projected US population in Jan 1 2023 is this and if we had to ask chargpt the same question it will tell us that it actually doesn't have real-time data and it was last updated back in 2021 so that means that our agent was able to determine that the model does not have this information so
[20:26] it went back to the tool set and realized that it's now got access to the serp API tool and it then Googled the answer and that's what it returned back to us so now we know how to create agents and assign tools to our agents next let's have a look at memory when creating our chatbot we do want our bot to remember previous conversations and we can use memory for that again I'm just going to clean up this code as we don't need most of this right for this memory section we will be using prompt templates so I'm just going to import a
[20:59] few things from Lang chain slash prompts and what we want to import for now is the chat prompt template as well as the system message prompt template and the human message prompt template as well so for our chatbot is create our chat prompt using the prompt templates so for this let's create a variable called chat prompt and we'll say that equal to chat prompt template and on chat prompt template we'll call this function from prompt messages this function takes in
[21:31] an array and the first thing we want to pause is the system message prompt template and on this class we've got this method called from template and it's in this method that will pass in the prompt and we'll use the system message to tell the model that we expect it to behave like a friendly chatbot so let's pause in a prompt like the following is a friendly conversation between a human and an AI the AI is is talkative and provides lots of specific
[22:04] details from its context so what this is saying is the chatbot will remember previous conversations and here we are telling it to pull specific details from our chat history in the conversations and let's keep going if the AI does not know the answer to a question it truthfully says it does not know that is quite a long prompt but that is prompt engineering for you so it is convention
[22:35] to always pause the system message first and that is to Prime our model then as per the previous example we want to pause in the human message which we can do by calling the human message prompt template Dot from template and in parentheses we can now pass the input from the user which will accept as a parameter by using the square prices but now we need to be careful because in a conversation we typically want to First pause in the system message then as
[23:05] we're talking to our chatbot the chat history will be built up dynamically and the human message that we paused here should be the very last message in the conversation and we are expecting the AI to respond to this last message but we saw Mal need to inject the conversation history in between these two messages so what Lang chain will do is it will automatically keep track of the post conversations so the chat history and it will inject that history in between these two items but in order for line
[23:38] chain to do this we need to provide a placeholder for the chat history so in order to use that placeholder we need to import message placeholder from the prompts package and now in between system message and human message we can pause in a new instance of the message placeholder class and in this class we need to specify a unique value which we'll call history and we'll get back to this in a second basically how this works is we will instruct Lang chain to make use of memory and we need to tell
[24:09] Lang chain where to find this history by providing a key value this could be anything by the way it's simply a reference to the position in memory where Lang chain can find these values so let's continue with the setup we will now create a new chain which is equal to the conversation chain which we also need to import so in our Imports I will import something from Lang chain slash chains and from chains we will import the conversational chain so in the past
[24:41] we use the llm chain like so but the conversational chain is like the llm chain with one addition it allows us to use memory so that our conversational bot or our chatbot can store the conversations in memory so back in our code heard the conversational chain takes in an object as input the first property is memory and at this point we need to create a new instance of the buffer memory object which is also something that we need to import so back
[25:12] in our Imports let's import buffer memory from Lang chain slash memory so back in our code we can set memory equal to a new instance of buffer memory buffer memory text an object as input with a property of return messages which will set equal to true then the second property is our memory key which is the key value that we pass into our messages placeholder and we call this history so simply copy this and add it to our
[25:42] memory key then we also need to pause in the prompt to our conversation chain and our prompt is equal to our chat prompt which we specified every year then lastly we also need to specify our llm which we haven't created yet so I just below our chat front I will create our new model I'll just call it model which is equal to a new instance of chat open Ai and I'll just pass in a blank object to use the defaults so for llm we can pause in our model and it save all of
[26:14] this so instead of using the llm chain like in the previous examples we are now passing in the conversational chain and as with the standard LL aim chain we passed in the lrm as well as the prompt but the difference between the two chains is that the conversational chain requires this memory object and this will store our conversation in memory so let's try this out on chain let's call the function call and let's pause in our prompt call text in an object and in this object we need to specify the
[26:44] variable names that we created in our prompt template so if I go back to our prompt template we call this variable input so in the call function I'll pass in the input property with a value hello from South Africa call returns a promise which we need to await and let's assign this to a response and let's go ahead and console log this response like so let's go ahead and run this in the terminal and after a second we get this response saying hello there it's great to hear from you in Sunny South Africa how may I assist you today awesome let's
[27:17] pass in another message to our model I'll just call this response to envo weight chain dot call and this time we'll pass it an input asking it do you know where I am and let's lock this response as well and let's run this in the terminal and this time it's saying as an AI language model I don't have the ability to know your exact location however I could tell that you are in South Africa based on the fact that you mentioned it in your previous message awesome so our chatbot was able to retain this information in memory during
[27:50] this session so we looked at memory in the previous video as well as in this video but please note that this is considered to be short-term memory after the decision ends the memory will be cleared and the model will lose context of our conversation as an example if I had to remove the second call to our chain and let's change the very first code to something like do you know where I am I'll save this change and at this point we are not telling the AI where we are we're simply asking it where we are
[28:21] and if we run this we can now see that the AI is saying that it does not know our current location and it can try and use our IP address to find our location but clearly the memory was cleared and it does not have context of our conversation anymore we will have a look at long-term memory within the series and in a nutshell long-term memory means storing this data in a data source like a database or files Etc let's also have a look at streaming as with the standard llm we can stream
[28:54] the response from our chatbot like so I'm actually going to delete most of the we will remove this chain as well as the buffer memory and also remove the spliceholder as well as the system prompt and the Jet prompt we'll only make use of the human message prompt in this example first I'll create a new model called Model I'll set that to new chat open AI chat open AI takes a few parameters what we are interested in is to say it's streaming to true we also have to specify the callbacks which is
[29:25] an array and with the Indus array or create a new object that triggers a function called handle llm new token which gives us a token as input and within this function we'll call Process dot standard out dot right and we'll pause the token to this write function and then we can awide model dot call cortex in an array or pass in a new instance of the human chat message and now that I think about it we actually
[29:55] don't want to use the prompt template here so I'll just remove this and instead I'll import the human chat message from Lang chain slash schema so then we will pause in new human chat message and we can simply pause our prompt into this and I'll use the same example as last time write a song about sparkling water let's save this and let's run this in the console and as you can see we are now streaming the response from the model thank you very much for following along and I do hope
[30:26] you are enjoying this series if you like these videos please consider subscribing to my channel and please like this video and please have a look at some of my other AI related videos on my channel thank you for the support I'll see you in the next one bye bye