Claude 3 Tool Use (Function Calling) Tutorial | JavaScript & Anthropic API
AnthropicAging📅 2024-04-12
Integration Flow
┌─────────────────────────────────────────────────────────────────────┐
│ INTEGRATION ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ USER REQUEST │
│ │ │
│ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ ────► │ │ ────► │ │ │
│ │ Claude │ │ API │ │ Database │ │
│ │ │ ◄──── │ │ ◄──── │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │ │ │
│ │ │ │ │
│ └────────────────────────┴───────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────┐ │
│ │ RESULT │ │
│ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Integration Points
- API authentication
- Request handling
- Error management
Transcript
[00:00] Anthropic recently added tool use to their Cloud 3 models, which is a huge deal. Tool use, also referred to as function calling, enables AI models to interact with their environment by utilizing external tools. For example, you might want to create an agent that is able to get the status of an order from a database, or perhaps send an email on your behalf. With the addition of tool use in the Cloud models, we can now supply a list of tools to the model, and the model will select the appropriate
[00:30] tool to complete its task. We could even provide a complex task to the model that would require several tools to be called, and the model will be able to handle this as well. This is an extremely powerful and important ability of the Cloud 3 models. So in this video, we will have a look at implementing tool use with the Cloud 3 models in JavaScript, but let me know in the comments below if you would like to see a Python version of this video as well. I will be building this project from scratch, so if you are not
[01:02] familiar with the Anthropic API, you will still be able to follow along. If you would like to download the source code for this project, then you can download it from this GitHub repo. I will link to it in the description of this video. Right, let's get started. Let's open up a new project in VS Code, and let's open up the integrated terminal, and let's instantiate a node environment by running npm init -y. This will create this package.json file, and in this file, and just below this main property,
[01:33] let's add another property called type with a value of module. Then let's install a few dependencies by running npm install. The first dependency is .env, which will allow us to store our Anthropic API key in an environment variable. Then we also have to install the Anthropic SDK by entering at anthropic -ai slash stk. And lastly, let's also install readline, which will allow us to
[02:04] get input from the terminal and pass that input as our message to the model. Let's press enter to install these. Great, now in our file explorer, let's create two files. First, let's create an index.js file, and secondly, let's also create a .env file. We will get back to this .env file in a minute, so we can close it for now. But let's add our logic to this index.js file. Let's first import something from the Anthropic SDK. So at the top of the code, let's enter
[02:35] import anthropic from this anthropic library. Then let's create an instance of this class by creating a variable, I'll just call it anthropic, which is equal to new anthropic and parentheses. Now this class does require an Anthropic API key in order to work. So there are two ways to populate this key. First, we can pass in an object into this class, as well as a property called API key. And from here, we can enter our API key.
[03:07] So because I don't want to expose my API key in the code itself, let's actually remove this object, and in our environment variable file, let's create a variable with this specific name anthropic_api_key. Let's set this equal to a string value, and let's go and find our anthropic API key. To get your API key, go over to anthropic.com, click on API, then get started now,
[03:37] and you will be prompted to register your account, and you will also be asked to verify your account, and for verifying your account, you will get $5 of free credits, which is more than enough to follow along with this tutorial. After logging in, click on get API keys, then click on create key, give your key a name, I'll just call mine youtube tutorial, and let's click on create key. Go ahead and copy your key, and then add it to this environment variable, like so. Please ensure to use your own key as I
[04:09] will delete this key after this recording. We don't have to do anything else with this .env file, so we can go ahead and close it. So we should be able to send messages to the anthropic API. Let's test that by calling anthropic.messages.create, and this takes in an object as input, and this object takes in a few properties, like the model name, which I'll set equal to claud3.icoo. I'm actually really impressed by this tiny little model. It's not only cheap to use, but it's
[04:40] actually very good at performing plenty of different tasks. Let's also set the max tokens to something like 1024, and let's also specify a property called messages, which is an array of messages. So this is typically like the user message, followed by the assistant message, chat history, context, etc. So let's start by passing in our very first message by adding an object, and these objects take two properties. So it's the role, which in this case is
[05:11] the user message, as well as the content of the message, which in this example is simply a string value, and it's hardcode something like hello. We will be grabbing this input from the terminal in a second, but this is hardcoded for now. Now this create method is an API promise, so we do have to await the response, and it's also assigned this response to a variable. Now all we have to do is console.log this response, and before we run this in the terminal, we do have to make these variables in the
[05:42] .env file available to this application. We can do that at the top of the code by running import all as .env from .env, and then all we have to do is call .env.config. So that will make this environment variable available to this code, which this anthropic class can reference. So let's run this in the terminal by calling node and index.js, and check that out. We've got our response coming back in
[06:14] this content property. Now before we move on to adding tools, I do want to convert this into a chatbot, so that we can have a back and forth conversation with it. This is entirely optional, and feel free to skip ahead if you want to, but I do feel that it makes the process of working with these models so much easier, and it's not too much effort to implement either. So firstly, let's add this logic to a function which we can reuse. So I'm going to create a new function called create API message,
[06:44] which is an async function, and let's also move this code into this function, like so. Then let's also accept the messages array as input, and then let's remove the array from this property, and let's simply call it like this. And then lastly, this function will simply return the response from this API request. It's also removed this console log for now, and what I want to do now is to receive the input of the user from the terminal, and then pass that to this
[07:15] create API message function. In order to get the user input from the terminal, let's import readline from the readline package. Then all we have to do is create a new instance of readline by creating a variable like RL, which is equal to readline.createInterface, and this takes in an object as input with two values, input, which is equal to process.standardIn, and output, which is equal to process.standardOut.
[07:46] We can get input from the terminal now by calling RL.question, and in parentheses, we have to pass in two values. First is a string value that will show up in the terminal, prompting the user to enter something, like user as an example. Secondly is a callback function, which will take the input from the terminal and pass it into this function. So let's create an anonymous function, like so. This function will provide us with the input from the terminal,
[08:17] and in this code, we cannot do anything we want with this input. So we could call our create API message function, and this function takes an array of messages as input, and it also returns a promise. So let's first await the response from this function. Let's assign it to a variable like AI_RESPONSE, like so, and in order to call await, we just have to change this anonymous function to an async function, and that will work. Then we can also console.log this AI_RESPONSE,
[08:49] and we can also make it a little bit fancy by adding AI to this response. Now, this create API message takes an array of messages as input, so let's actually create that array. So let's set const messages equal to an array. Let's create that object. So in the object, we have to specify the role, which was user along with that content property, and we'll set content equals to the input from the terminal. So let's pass messages to our
[09:19] create API message function, and in theory, this should work. So in the terminal, let's run node index.js. We can now see that it's prompting the user for input. Let's say, hello, and we also get the AI_RESPONSE coming back. So there's two things I want to change here. First, for the AI_RESPONSE, I don't want to show all of this. I only want to show the final response, which is sitting in this text property over here. Second, it doesn't matter what else we type. If we press enter, nothing else happens.
[09:51] So we do want to call this functionality in a loop, and there's also no nice way to exit out of this. So all you can really do is press Ctrl C. So let's make a few improvements to this code. In order to call this code in a loop, we can do the following. Let's wrap all of this read line logic in another function. So let's create a new function called startChat, which doesn't take in any parameters, and let's add all of this logic to the startChat function.
[10:22] So what we can do now is after we write the AI's response to the screen, we can simply call startChat again. So this means this function will effectively call itself in a loop until the user terminates this process. Let's also give the user an elegant way to exit out of this process by entering the words exit. So next to this user prompt, let's also add something like type exit to quit, like so. So in this code, we can simply say if
[10:52] input.toLowerCase is equal to exit, then we want to do two things. First, we want to close read line, and second, we just want to return out of this function. And lastly, let's only get the text from the response by going to aiResponse.content and we'll simply grab the first result in this array, .text. So finally, all we have to do is call this startChat function at the bottom of our
[11:22] code to start this process. Let's test this in the terminal, and now we can see the user gets a nice little message to exit out of this process. Let's type hello. The aiResponse is coming back in a nice format now, and we are able to continue with this conversation, like tell me a joke, and we get a response back. But more importantly, we now have a functional chatbot in our terminal, and if we want to exit out of this, we can simply type exit and enter, and we were able to terminate this conversation.
[11:53] Now let's move on to the real reason you're here. Let's have a look at adding tools to our models. So first, let me explain a few concepts about adding tools to models. And let's say we wanted to ask a question like, "What is the current weather in New York?" The model will give you some sort of answer in the lines of, "It doesn't know because this data is not available in its training data." Or if we ask a question like, "What is the current status of order 1?" The model will also be exceptionally confused by this.
[12:23] So what tool use allows us to do is to provide a list of functions or external tools that the model can use to achieve its task. So if we ask it what the status of an order is, the model will first check if it's got the answer in its training data or context, and if it doesn't, it will see if there are any available tools in its toolkit to fetch this information. So to start off with, let's create a tool for fetching the current weather. Let's enter out of this conversation and
[12:55] let's get back to our code. Let's create our get weather tool. To keep this code clean, I'm actually going to create a new file and I'm going to call it tools.js. Now tools are created in two phases. First, we need to provide a JSON schema which is used to define the definition of the tool as well as its inputs. The second phase is the actual code that will be executed by that tool. Let's start with this schema definition. So I'm actually going to
[13:26] write a comment called schemas. Let's create a new variable called weather tool schema. And this is equal to an object and this schema is taking a few properties. First is the name of the tool, which we can call get weather. And it is important to follow this naming convention with lowercase characters and underscores in between the words. We also have to supply a description and this description is extremely important.
[13:56] The model will use this description to determine when this tool should be used. Let's enter something like get the current weather for a city. Then we also have to supply the input schema and these are basically all the input values that needs to be passed into this function. This input schema has a type of object and we also have to supply a property called properties. And this is where we can define all the input parameters for our function.
[14:27] Let's keep this simple and only pass in a single parameter called city. And for each of these parameters, we have to pass in an object which contains two properties. So it's the type and in this case, we are expecting a string as well as a description. And this description is extremely important as the AI model will use this to figure out how to extract these properties from the user's prompt. So if I say something like give me the
[14:57] current weather in New York, the model will use this description to figure out what information to extract from that prompt and map it to this city property. So in this case, this is simply the name of the city for which to fetch the weather. The final property that we have to specify is the required property, which is an array of values. And here we can simply specify all the parameters that are required for this function to work. Now that we have the schema defined,
[15:29] we can provide the schema to the model as a tool. So let's also export this schema from this file so that we can import it in this index file. But before we do that, let's also add the function that will be executed for this tool. So I'll add a comment called tool handlers. And let's create a function called weather tool handler. This will receive a bunch of parameters from the model. So let's simply call this input. And for the logic, let's simply console
[16:00] log weather tool handler along with the input that was passed from the model. And now we can implement any fancy logic that we want. We can call API endpoints or do whatever we want to do. Once we have the result, we can return the result as a string 26 degrees Celsius or whatever you want. So now that we have our schema as well as the logic that should be executed by that tool, let's go back to our main file. And now let's have a look at how we can
[16:31] pass tools to this anthropic API. It is worth mentioning that the tool use functionality is currently in public beta. So calling the API will look slightly different depending on when you watch this video. In this video, we will be accessing this tool from the beta namespace in the SDK. But you might be able to access it directly by the time you watch this video. But don't worry though, it's not too confusing. All it really means is on this anthropic package, we have to call
[17:02] .beta.tools.messages.create. So if this doesn't work for you, try omitting .beta or .beta.tools entirely to see if that works for you. This works exactly the same way as we added previously with one addition. We can also pass in a tools property which is an array of values. So what we can do is we can pass our schema definitions into this array of values.
[17:33] So let's import something from .slashtools.js. And from this, let's import the weather tool schema. Let's copy this, let's add it to our list of tools. And now when we interact with this model, it could decide to use this weather tool when we ask it questions about weather. Let's see what that looks like. So let's also add a console log. And for this, I'm actually going to add something like debugging and let's return the response like so.
[18:06] Now watch what happens when we execute this. Let's ask what is the current weather in Johannesburg. So we got the response back from the assistant, but you will also notice that the stop reason is tool use. And within the content property, instead of receiving the text response from the model, we get something that looks like this. This object has a type of tool use along with an ID for this tool, the name of the tool, which it wants to
[18:37] call and the input for this tool. And then lastly, we are simply getting the AI response as undefined, as we didn't get a text response back from the model, but instead the model expected us to first execute this tool, get the response from the tool and then pass that back to the model. So let's change our code slightly. We now know that simply calling this API message function and getting the response back is just not enough. We first have to interrogate that
[19:07] response to see if we need to call any tools and then execute those tools, get the responses and then pass them back to the model. So let's improve our code a bit. Instead of calling create API message directly in start chat, I'm actually going to create a new function and I'm going to call this process conversation, which is an async function. This will receive the input from the user as a parameter and in its logic. Let's do the following.
[19:38] First off, it is this function that's responsible for calling the API message. So I'm actually going to replace create API messages here with process conversation. And instead of passing in the messages array, I'll simply pass in the input from the terminal. Let's also cut this messages array from this logic and let's move it to process conversation and we'll simply add message to this content property. So after this messages array,
[20:10] let's create a variable called response, which is equal to await create API message and we'll pass the messages array to this function as input. So far, nothing too fancy. Start chat will simply get the value from the terminal and exit if the user entered exit. Otherwise, it will call process conversation, passing in the user's input. And finally, it will log out the response from the AI. And this is the final response and not
[20:41] the tool responses, which we saw earlier. And then it will continue with this loop. Now let's continue with process conversation. Now, when we get a response back from the AI, we want to see if we need to process any tools. So what we want to do is interrogate this response to see if the stop reason is tool use. And if it is tool use, we need to call these tools in this content block. And as I've mentioned earlier, the model is able to pass more than one
[21:13] tool into this content block. That is why this is an array. So in our logic, let's do the following. While response.stop reason is equal to tool use, we need to do the following. And it will make sense in a minute as to why we are building this while loop. But effectively, for as long as the model is telling us to call tools, we will simply keep calling those tools until the model is happy. We actually need to change this from const to let,
[21:44] because we will be changing the value of response within this while loop. So if the stop reason was tool use, we now need to loop through these items in the content block to fetch the details around the tool that needs to be called, like the get weather tool in this example. So let's enter for let i is equal to zero, and i is less than response.content.length and will simply iterate
[22:14] through each of these tools. So in this example, content length is just one. So we will only run this for loop once, calling that get weather tool. So what we can do now is extract all the information we need about that tool from this property. So that's the type id name, as well as the input parameters. So let's do that by typing const. content type is equal to response.content and we'll grab the value from the i position.type.
[22:47] Then let's also get the tool name, which is equal to response.content i.name. And if we don't find a value, we'll simply default this to a blank string. Let's also grab the tool input, which is equal to response.content position i.input. Or it can default as null. Let's also grab the tool id, which is equal to response.content i.id.
[23:20] Or it's default to an empty string. So we've now effectively extracted all of this information from the content block. Now we just have to decide what to do with this information. So first I would say that if the content type is equal to text, then we can simply log the ai response as response.content position i.text. And the reason for this block is if we have a look at this content block, right?
[23:51] This is an array which can accept multiple objects. And at the moment, we only have one object with a type of tool use. So you might be wondering why I'm looking for a content type of text if we only receive tool use. But this is from experience. I've had situations where the model would provide both a tool use type along with a text type in this response. For example, we might receive this get weather object, but we might also receive a text object
[24:22] with something like, "Okay, I will get that for you." Or, "Please give me a minute." So this code will simply ensure that we write out any messages from the model during tool execution. And after writing out console log, let's also continue with the for loop. So we can also add an if statement that says, if the content type is equal to tool use, then we can go ahead and execute our tool functions. So we could simply say something like,
[24:54] "If tool name equals get weather, then we can call that get weather function." Let's actually do that. Let's also export this weather tool handler function. Then back in index at the top of the code, let's also import the weather tool handler function from tools. And then if the tool name equals get weather, we can simply call that handler function, and then we can pass in our tool input. Let's also assign this to a variable
[25:26] called tool result like so. So now that we have the result from the tool, we somehow have to pass that back into our model. So effectively, what we need to do is after we've executed all of these tools and collected their responses, we have to call the create API message function again, passing in an updated version of the messages array. So in our function, just below the for loop, let's just go to the end of this for loop. So right over here, we need to call our
[25:58] create API message function again. Passing in a messages array. Of course, we still have to await this function call and we'll assign the response to the response variable. And this will give us a brand new response from the AI model, which we can process. And that is why I'm calling this within this while statement, is at this point, we are calling the API getting a new response. We will then have a look at this while statement. And if the latest response still has a
[26:31] stop reason of tool use, we will simply continue with this logic and call the API again. And we will keep doing that until the agent no longer needs to call any other tools. So finally, outside of this while loop, we can simply return the final response from the model, which is response.content position zero dot text. Or if we don't have any value in that field, we can simply default to no response.
[27:02] So now the question is, how do we get this tools result into this messages array? Because our messages array only has one object in it. And that was our initial user message. So what we need to do is also append the response from the AI model and then also append the results from the tools. Thankfully, that's very easy. So first after the response, it's simply called messages dot push. And we'll pass in an
[27:32] object with a role of assistant. Then second, we have to pause in the content, which is response dot content, like so. So this is basically building up a chat history, which contains the initial message from the user. Then it also includes this response from the model, where it's instructed us to use this tool with this ID with these inputs. Now we will also pass back a user message containing the results
[28:02] of these tool use calls. So back in our code, just below this tool result, let's append this result to the messages array. Now this is super important, as I've seen other tutorials get this wrong. And it can be extremely frustrating to deal with. You might be tempted to do something like messages dot push, and with an object with a role of user. And then you might want to append the content as the tool result as an example.
[28:35] The issue is that if you were processing more than one tool, you will append more than one user message next to each other in the messages array. And you will receive error messages saying that you cannot have more than one user message following each other. So instead, what we need to do is change content into an array of values, and each of the tool results will be added to this array as its own object. So let's see how we can do this. Outside of this for statement, I'm
[29:06] actually going to create a new variable called tool messages, which is equal to an object with a role of user and content, which is an empty array. This means that in our code, we can now call tool messages dot content dot push. And now we can pass in the results of our tool. First, we need to set the type as tool result. Then we also have to specify the tool use ID.
[29:39] And this needs to be the same as this ID property assigned to this tool use. So we call it tool ID. Then lastly, we can specify the content, which is the result from the tool, which was tool result, like so. So this will append our tool results to this tool messages object, and more specifically, the content property within that object. And finally, we can append the tool messages to the final messages array.
[30:11] So after the for loop, we can call messages dot push. And let's add our tool messages object. Let's also console log messages to see the final output. I'll also add something like final messages. Now, this was a lot of work. So let's go ahead and actually test this out. In the terminal, I'm actually going to clear this. Let's run this again. And let's ask it, what is the current weather in Johannesburg? So let's have a look at what we got back. In debugging, we can see that we do get
[30:43] the stop reason of tool use. And in the content, you can see what I meant earlier. We get two objects back, one with a type of tool use, which contains our get weather function. But we also get this object containing text. Then we get this response back from the model saying, year is the current weather for Johannesburg. We can see that the input into the weather tool contains a city as Johannesburg. So it was able to correctly extract the city from our prompt.
[31:13] And this is what our final messages array looks like. So first, we have our initial prompt, then the response from the assistant, and then finally, a user response containing the response from the individual tools. And then finally, we can see over here, we do get the final response saying, the current weather in Johannesburg is 26 degrees Celsius. And that is the value that we returned from our function. And we have a stop reason of end turn.
[31:44] And we also get this error message. And let's have a look at how we can resolve this. All right, so this error is actually just caused by this console log in this start chat function, because AR response is simply a string. So we can just remove dot content and dot text so that we are only left with this AI response value. So let's try this again in the terminal. Let's ask what is the weather in Johannesburg. And this time we get the response back saying that the current weather in Johannesburg
[32:16] is 26 degrees Celsius. And that is how easy it is to add custom tools to your AI applications using tool use in Cloud 3.