GPT-4 API in JavaScript for BEGINNERS - Visual Input
OpenAIVintage📅 2023-03-18
Integration Flow
┌─────────────────────────────────────────────────────────────────────┐
│ INTEGRATION ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ USER REQUEST │
│ │ │
│ ▼ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ ────► │ │ ────► │ │ │
│ │ OpenAI │ │ API │ │ Output │ │
│ │ │ ◄──── │ │ ◄──── │ │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │ │ │
│ │ │ │ │
│ └────────────────────────┴───────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────┐ │
│ │ RESULT │ │
│ └───────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Integration Points
- API authentication
- Request handling
- Error management
Transcript
[00:00] hello and welcome back to another video in this video you'll learn how to include gpt4 API in your own JavaScript project so gpt4 is definitely an upgrade on GPT 3.5 but for the most part it seems very similar so apparently it's way more creative in its responses and it is able to handle larger prompts and produce longer output so one of the new features is to provide visual input into the model for example images and we can then ask the AI to describe the image or
[00:31] to translate text in the image Etc so for this demo I created a very simple web application which will accept a URL to an image and we can then provide the prompt for example we can ask the AI to describe the image for this demo I generated this image using merge Journey but you can use any image that is available online so for example I'll copy this URL paste it in our application and I'll just add a prompt for tell me what you see in the the following image let's give this a spin
[01:02] it will take a few seconds to complete we can see the response the image features a beautiful red-headed woman wearing a green dress she has a fair skin tone with freckles and her hair is long and wavy so it's doing a very good job of analyzing this image this ask it some specifics like what is the color of her hair and it's saying that and it's saying that the hair is red what color is her eyes and is saying that she's got green eyes what color is her dress and
[01:34] it's saying that she's wearing a green dress so although this demo is very simple there are a lot of use cases for this so if we go back to the documentation you can see the example that I used where they provide an image of whole bunch of ingredients and in also AI to tell us what recipes we can make using these ingredients so enough talk let's bought this application right so the first thing you need is you need to create an account on open AI you can do this by going to platform.openai.com and then create your
[02:04] account at the time of making this video the gpt4 API is only available if you join the waitlist if that's still the case all you need to do is click on join the API waitlist and then enter your details here after a few days you should see an email like this saying you can now access the gpt4 API once your access has been approved you can test it out by going to platform.openi.com and then going to the playground tab in playground you can find the gpt4 model under chat and you
[02:36] should see it every year so how this works is the user will provide a prompt from the front end and our prompt will be structured in a certain way so first We'll add the text that the user entered followed by the URL that the user provided so it will look something like this and after running this we'll get this response from the AI I've seen a lot of videos and articles where the developers run into this issue where when you run this prompt the AI might respond with something in the lines of I'm an AI model and I'm not able to process images so I found that if we add
[03:08] the following prompt to the system it will reduce those type of issues so in the system at this prompt use your ability to analyze images to accept an image URL and then assist the user with their prompt let's just test this out again describe the following image and let's run this all right perfect right so let's get started with our application for this demo I'll be using next JS because we need server-side logic as well as our front end but you can really use whatever framework you
[03:38] want feel free to create an Express server or use wealth kit or whatever you prefer right so first open up your IDE and create a new folder then in order to initiate the new next GIS application run npx create next app add latest followed by a period would you like to use typescript I'll say no eslint now a source directory now and we'll use the experimental app directory if we import aliases just press enter for styling the application we'll use Tailwind CSS so to
[04:11] set this up I'll click on get started framework guides and I'll click on next JS we've already executed step one so let's do step two you can just copy these commands back in vs guide I'll just paste in those two commands and press enter for step 3 we need to add the following content to the televan.config file so back in our project I'll open up telwyn.config and replace the content with this we then need to add these three lines to the globals.css file so back in our project
[04:41] under app we can open up globals.css and replace it with those three lines I'll just delete any files that I don't need so I'll delete page.module.css and in the page.jis file we can delete all of these Imports and I'll delete everything from Main to the closing main tag we'll just add back in this main tag with the header of hello world let's run our server by running npm run Dev you should find the link to the local Dev server in the console and
[05:12] after opening up this we can see Hello World so our Dev server is working before we build the front end I'd like to focus on the back end logic first because we need to run the open AI Logic on the server we need to implement our own API first we can create an API within the app folder and within this API folder next year is has given us the sample function which is a get method and all this does is it returns hello in xjs we can test out these API endpoints by using a program like Postman so what
[05:43] we can do here is enter the URL and Then followed by the name of the directory like hello in this route file we only have a git method available so I'll set the verb here to get let's run this after running this we do get that response back hello nextjs right so for our application I'm going to rename hello to open AI I'm also going to change the get to a post let's save this and in Postman I'll change the method back to post and let's run this okay that won't work because I also need to
[06:14] change it to open AI let's run again and that works right in order to call functions on open AI we need to install the following package so in the console let's run npm install openai also in order to call functions on openai we need to provide open AI with an API key we can get the API Key by going back to our openai platform you can click on personal view API keys and you could then click on create new API key you can
[06:44] then copy this key and go back to our project right so in our project we can either save this key in the method itself but this key is private and it needs to be predicted so it's recommended to store this value in an environment variable file so in in the environment fall we can now create a new variable called openai API key and we can paste in the key value please ensure to use your own key I will please ensure the user on key I will be deleting this one after the video then back in the route file we need to import a few
[07:16] things from openai so we can import configuration and open AI API then in our post method we first need to create this configuration object by instantiating the configuration clause and then we need this special and then we need to specify the API Key by referencing the open AI API key value in the environment variable file we can then instantiate a new object for openai and then pass in the configuration object as input we now have access to
[07:48] the open AI functions using this openai object I'm going to wrap this logic in a try catch block just to deal with any exceptions that we might get back from calling open AI if we do receive any errors from open AI I do want to return them as a new response and we'll simply return the error text so we'll pass back an object with an error parameter and the value will be error Dot message this object also needs to be Json stringified so I'll call Json Dot stringify and then
[08:19] wrap this object in this stringify function right this add the logic to the try block first I'm going to just move this return statement to the try block and now in the try block we can call a function on open AI so we'll call openai Dot create chat completion just a note we will not be calling create completion like with GPT 3.5 instead we'll call the chat completion function if we go back to open AI for a second to the playground we can see that on the completion functions we do not have
[08:50] access to gpt4 but it is available on the chat completion and that is why we're calling the create chat completion function in our code the create chat completion and function takes in an object as input the first value that we need to specify is the model that we want to use in this case we're using gpt4 and we also need to specify the list of messages so messages is an array with objects the first message in this array will be the instruction from the system role so back in open AI it is this role over here so for this we'll
[09:21] specify a role of type system and for its content or provide the following text use your ability to analyze images to accept an image URL and then assist the user with their prompt then for the second message in this list we'll create a role of type user for the content we'll pass in the value of the prompt that the user specified this will be this message here so this is the user role and then we'll specify the text like analyze the following image with the URL and this will be a value that we
[09:52] get back from the user and this will be a value that will be passed in the body of the request message so we need to X extract this prompt value from the body of our request at the top of this function I will destructure prompt and we will get that from the request object dot Json we can then take this prompt value and pause that in as the content for the user prompt the create chat completion function returns a promise which we need to await and this assign that to a response variable once we get
[10:22] a response back from openai we want to pass that back as the value in this API call we will also Json that stringify this result we'll pass in an object with a response attribute and the value of this attribute will be the response that we got back from openai dot data dot choices and in this case we will only receive one choice we should be able to test this out so let's start up the dev server if it's not running let's go back to postman and let's try to call this so
[10:52] in Postman I'll just go to body or go to Raw so I'll pass in a Json request so just ensure that you select Json from this list will open the request with these brackets and then we'll pass in a value for prompt and let's give a value like hello there and let's run this and after running this we can see the response coming back from open AI so in the response we've got this message object within message we get the role for assistant which is the AI just to show you on open AI what that looked
[11:23] like so after running this we get this response back from the assistant role with the content of the message so that is what we're seeing here so we're getting a response back from assistant and it's responding to our request or fellow excellent that is it for this API now let's call our API from our front end application so back in page.js let's do the following the first thing that we need to do with this file is since next.js is server components by default we need to add this one line to the top
[11:53] of the file to make it accessible to users so we'll just add this line for use client I'm going to add a few few classes to main so I'll add class names or container margin top of six mxr2 for centering the content in the middle of the page padding left and right of four Max width of 2XL will align the content flex and flex column for vertical alignment and a gap of four just to create some spicing in between the content let's refresh the app so we can
[12:23] see Hello World showing up with some margin top and some padding on the sides this changes from Hello World to gpt4 image analyzer let's add some styles to this header tag We'll add a text or 4XL and a font of bold So Below the header we'll then add the form so just to show you what I'm referring to we'll add two input Fields as well as this button so let's add the first input field so I'll add input as a self-closing tag I'll give input a few classes as well so for the input We'll add padding of top and
[12:55] bottom of two padding left and right of two we'll tell it to take up the 4 width with the background of white will create a border with the Border color of Christ 700 will give the placeholder text a color of gray 700 and will round the corners as well then I'll give this on placeholder text like enter and image URL and in order to reference the values in this field we'll import the package from react and specifically the use ref package from react we containing create an object to link this to so I'll create
[13:27] a variable called URL riff and I'll make that equal to use ref so let's go back to the input field We'll add a property called ref and we'll assign URL ref to this for the prompt I'm just going to copy this input field let's change the stakes to enter a prompt we'll create another reference field we'll call this one prompt rev and we'll assign prompt drive to this field lastly We'll add the button we'll give the button some text like analyze and we'll give the button
[13:58] some Styles as well we'll set the text as uppercase vertical padding of two horizontal padding of two or rounded Corners we give it a background color of lime 700 and we'll set the text to White this also adds some styles to the form tag We'll add flicks Flex column for vertical alignment and a gap of two great okay so on the form let's add an on submit event handler and on the onsubmit event handler we'll call handle submit is create this function so we'll
[14:29] create this handle submit function which takes the event as input and the first thing it needs to do is call event dot prevent default this will prevent the page from refreshing when we click on analyze because we'll be working with the fetch API we need to convert this function to async then what we want to do is call the fetch API and we want to call the API that we just created and we can find that on the API slash open AI we need to specify some additional attributes like the method which needs
[15:01] to be post we need to specify the headers which takes in a content type of application Json and in the body we want to pause in the prompt and the prompt will be a concatenation of these two Fields so the first part of the prompt will be the promptrev dot current dot value followed by the URL ref.current dot value the fetch API returns a promise which we need to await and we also want to assign this result to a variable called raise after the API call
[15:32] we just want to perform a check to see if race dot OK is successful so we'll check if restart okay is not successful and if it's not we'll just write a new error with the text unable to fetch response so what I also want to do is just wrap this logic in a try catch block so at the top of this I'll call try I'll move all of this code into the try block and after try we'll call catch which receives the error object for now we'll just write any error messages to the console after checking if race is
[16:05] okay we need to convert the response to something that we can use so I'll create a new constant called data and I will then await restore Json it's console log the value of data for now let's test this out in the browser I'll open up the developer tools open up the console and let's try this I'll grab the URL of this image not paste it here and in the prompt I'll just write something like describe the image let's run this okay so when running this we are getting an error message and that is because in our
[16:35] logic when we pause in the body we can't pause in the object like this we need to pause it first so I'll just call json.stringify and we'll wrap this object in the stringify function let's save this I'll clear out the console and let's run this again after a few seconds we get this response back and if we open up this response object we can see the message coming back from the the assistant as well as the description of the image so it just seems to be working so let's show the image and the response
[17:06] on the page so back in our code we need to write the value that we get back from openai to a state variable and then we'll write out the value of the state variable in the HTML so to do this we'll also import use state from react and I will then create these states so the first one is for the image I'll just call it image and the seter function I will call set image that is equal to use State and initially the value will be null the second state is called text
[17:38] with the setter function of set text and on this we'll call you state and the initial value will be a blank string the last state that I want to create is called loading with the seter function of set loading with an initial value of false we'll set loading to True when we call openai and we'll set it to false once be done this is just so that we can have some feedback on the screen that shows the user they'll be waiting for feedback from openai so in our handle subnet function the start of our code
[18:09] will set loading to true and then after we've called open II or after we've dealt with any exceptions we want to set set loading to false so for this I'll also add a finally block to the try catch so whether we get an exception or processing was successful finally will be triggered and then finally we will call set loading and we'll set that back to false we will also set the image as URL ref.current dot value and we also want to set the text to an empty string
[18:41] then after we receive data we want to call set text to data dot response Dot message dot content then I want to create a Dev for showing the image so I'll create a div within Dev or create an image tag which is self-closing and source of the image will be the value of the image State variable then below the image we want to create another div which will restore the text that we got back from open AI for this I'll add a header as well with the text of response
[19:13] let's give this header a few Styles like text 2XL and font semi-bold then below this header I want to write this content conditionally so if loading is true then I want to return a P tag with the text analyzing else if loading is false in other words we got a response back then we'll write out the P tag and the content of this will be the value of text I also want to give the analyzing P tag a style so I'll add class name the store that we want to add is animate
[19:45] pulse so what this will do is while analyzing is shown the text will just pause to indicate that it's loading right let's give this a spin so I'll just copy this image URL again and paste it here and for the prompt I'll add the text describe the image and let's click analyze the image is showing up and below the image we can see the response and the text analyzing and once this is done we can see the response from the AI if you want you can leave it like this I do want the image and the text to be next to each other so what I'll do is
[20:16] I'll actually wrap the image and the text in another div and I'll give this wrapping div some Styles I'll add grid as well as grid columns too and I'll give some spacing between the two elements with a gap of four awesome after saving this we can see the image as well as the response let's try another prompt like what is the color of her hair let's try this and pick it out response right guys this was a very quick demo on how to use gpt4 in your applications if you like this video
[20:48] please consider subscribing to my channel where I do a lot of videos and content based on AI and please like this video as it will help this channel grow I'll see you in the next one I