How to Run AI Jobs in the Background (Claude Code + Inngest)
How to Build Full-Stack AI SaaS with Claude Code (Tutorial)Fresh📅 2025-11-18
Tutorial Overview
┌─────────────────────────────────────────────────────────────────────┐
│ LEARNING PATH │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ LEARN │
│ ───── │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Watch │───►│ Understand │───►│ Follow │ │
│ │ Tutorial │ │ Concepts │ │ Along │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ PRACTICE │
│ ──────── │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Try It │───►│ Experiment │ │
│ │ Yourself │ │ & Adapt │ │
│ └─────────────┘ └─────────────┘ │
│ │ │
│ ▼ │
│ MASTER │
│ ────── │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Build │───►│ Share & │ │
│ │ Your Own │ │ Iterate │ │
│ └─────────────┘ └─────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Key Takeaways
- Core concepts explained
- Step-by-step implementation
- Practical examples
Transcript
[00:00] All right, so we've made some good progress with our real world SaaS app, but there is one major concern. When users generate these plushy images, it can take about 30 seconds or a minute for this generation to complete. So what happens when the user actually closes the browser or refresh the page and decide to just start a new generation? What happened with that previous run? And how would we know if that execution actually failed because of some technical error? Now, debugging these types of issues can be a real pain. You
[00:30] basically have to go and dig through some server logs that could be hours, days, if not weeks old. Ideally, what needs to happen is those generations need to run in the background, in a safe environment that's also observable, so we can see exactly what's going on. So in this video, we'll add ingest to our application. This will take our app from being a POC or a toy app way closer to being production ready. So as with everything else in the series, ingest is open source, so you don't have to pay a thing. You can
[01:01] definitely self-host this platform yourself. And I'm not sponsored by any of these platforms. These are just tools that I ended up using in my actual applications. In fact, let me show you what this looks like by showing you another app that I bought. So this app works very similar to the one that we're building in this tutorial. You upload an image, so it could be an image of any room, and then we can choose a prompt to make changes to this room. Now, let's fire off generate. So while this image is generating, we can actually go over to ingest, and we
[01:34] can see this new run. If we expand this, we can see all the individual steps within this run, including the image generation. We can add all sorts of steps to these executions. And of course, we can decide to log whatever we want in the output of each step. If any of these steps failed, they will clearly be marked in red, and we'll be able to view the logs to see exactly what's been wrong in that step. After we resolve the issue, or maybe it was just a temporary issue like the network was down, we could take on rerun to restart
[02:05] this entire process. Now, ingest comes with a lot of very cool bells and whistles. Firstly, it's very easy to implement into Next.js applications, but I do want to show you some other really cool features. If we go to ingest functions and flow control, we can see a lot of really cool features offered by ingest, like concurrency control. And we can use this to determine how many of these functions should be allowed to run in parallel. So we might want to reserve resources for certain heavy processes.
[02:37] We can also easily implement things like rate limiting, debouncing, and under errors and retries, we can also tell ingest to automatically retry runs if they actually fail. Right, so now let's have a look at implementing this into our app. Unless you're vibe coding, I highly recommend reading the documentation to understand exactly what libraries and functionality you're adding to your app. So in the documentation, I'm definitely interested in this Next.js getting started article,
[03:08] because we are using Next.js in our app. So I'm actually going to copy all of the sticks and back in our project under docs, I'll simply create a new subfolder called ingest. And in there, let's create a new file and let's call this Next.js getting started dot MD, and it's pasting everything that we just copied. Then what I'm also interested in is this install the SDK page. So I'm going to again copy all of this and add it to our documents
[03:39] as well. All right, so I think that's good enough for now, actually. Now, of course, you could use MCP servers like context seven to get the agent to retrieve the up to date documentation itself. And we will do that in this video as well. But the reason I like manually capturing the documentation is because some of those MCP servers are not available in Cloud Code web. And the online search tool is not all that reliable either. Some web pages do not allow the agent to scrape its content. So this is just a
[04:10] really convenient way to pull the documentation into the project structure, so the agent can access it from anywhere. Alright, so let's pull up Cloud Code, let's go into planning mode. And let's say, hey there, please create an implementation plan for adding ingest to our application. Currently, when a user generates an image, that function is running in the background without any observability. So what we want to do is use ingest to monitor these image generations. So when the
[04:41] user clicks on generate, it should create a new run in ingest. And that run will be responsible for generating the actual image in the background. Please also consider things like rate limiting, concurrency, and implement retries. I've added ingest documentation to the docs folder. Also feel free to use other tools like web search or context seven to retrieve up to date documentation for adding ingest to this project.
[05:12] Alright, so I'm just going to clean up some of these typos because ingest actually contains two ends. Okay, so God is asking us a few clarifying questions. Like when should credits be deducted from the user's account? So I actually like option three, we will reserve the credit as soon as user clicks generate, but we'll only deduct on success. If the generation fails, the credit actually won't be deducted. Let's go with that option. How should the user be notified when the generation is complete? So we
[05:42] could simply pull the status of the generation every few seconds, or we could get real time updates with server sent events, or something like an email notification, I'll just go with option one, what should happen to a file generation in the database? So we could keep the record with a file status, or we could keep and just hide it from the gallery or delete automatically. So I think I'll just go with option one, let's just be transparent. Otherwise, our users might wonder what's going on. How many
[06:12] generations should the user be able to run simultaneously? So one generation at a time, three to five concurrent generations, or unlimited? I think let's go with option two, or let's submit our answers. And we'll see what cloud code comes up with. All right, so let's have a look at this implementation plan. I'm actually just going to go through all of this and make sure that I am happy with this plan. So I do want to note a few things, we actually have to create two new environment variables. So we must not forget to do that. Right,
[06:43] so I can also see that we're going to make database changes, which involves extending the user table with another field to store the reserved credits. And then we're also going to extend the plushie generation table with a failure reason field, a processing started ad field and ingest run ID. Now I actually don't like this failure reason field. Instead, I would prefer if we just have a status field, and we'll actually be able to view the failures within ingest itself. So I'm just going
[07:14] to say let's go back to planning, I'll just paste in that text and say for the plushie generations table, wouldn't it make more sense to store the actual status of the run, we will be able to view the failures within ingest itself against the runs. So we'll be able to debug it that way. But I think in the plushie generation table, we just need a status field. All right, let's send this. So really, at this point, don't be afraid to have a back and forth conversation with Claude, and to ask any
[07:47] terrifying questions. If you're uncertain about something, ask Lorde Cote to give you different options. Right, so apparently we already have a status field. So we actually don't need this field. So the reason Claude wanted to store the failure reason text was because it wanted to show it to the user. But that actually won't make sense because for the most part, those failures will be technical, and it wouldn't make any sense to the user. So instead, we'll just show some very simple text. So I'll just go with option
[08:18] one, which is this thing over here, no error messages stored. In fact, this might Claude reconsider the reserved credits field as well. While we actually don't have to reserve the credits in a dedicated field, we'll simply count the amount of processing generations. So with all that said, let's go into edit mode, and let's select the create feature command. So what this will do is create a new subfolder within this specs folder. In fact, we can see it every year. It's called ingest background jobs. And within a
[08:50] second, Claude will create a requirements document and a detailed implementation plan. And while this is running, I will really appreciate it if you could hit the like button and subscribe to my channel for more Claude code content. Also, I would love to hear from you in the comments. Are you using Claude code or have you switched over to something else like cursors compose one model? Or have you tried GPT 5.1? Let me know. All right, cool. So we've got our implementation plan and our requirements document. This implementation plan is split into phases with actionable tasks per phase.
[09:23] This is really useful for keeping track of our progress. So now that we've got this implementation plan, what we can do is clear the chat and finally start with the implementation. All we have to do is grab this folder. And of course, we could ask Claude to try and implement everything in one go. But the most important rule with the gente coding is to keep the context window as lean as possible. So instead of overwhelming the agent with asking it to implement everything, we'll implement this in smaller chunks. So
[09:54] let's say, please implement phase one to phase three in the implementation plan. Once you're done, mark the tasks as complete in the implementation plan. That is cool to me. It's in it. And we can see as Claude is working through the solution, it's sitting these tasks as complete and phase three is complete as well. Alright, we're now ready to move on to phase four. So really, all I'm going to do is clear the chat, pull in the specs folder and say, hey,
[10:24] please go ahead and implement phase four to phase six of the implementation plan. Once you're done, mark the tasks as complete in the implementation plan. And I'll keep going through this process until all the phases have been completed. I'll see you then. Alright, so Claude code just went through everything. So it's implemented all of the phases. And all it's safe now is to set up our production environment. We'll get back to this step after we've tested this in dev. So what I like to do is to clear the conversation. And then for
[10:55] the last time, pull in that same feature folder. So we've implemented all of the phases in this implementation plan. Your job is to review the solution to ensure that it's following good coding practices. That's in lined with ingest best practices as well. You can use the documentation in the ingest folder, along with tools like the contact seven MCP tool to retrieve up to date documentation for setting up and using ingest, please also
[11:25] resolve any issues that you find. Cool, it's in this. Well, it's a cloth finish the code review and everything looks good. Now before we test our app, we do need to start the ingest dev server. So what we can do is open up a new terminal session. And to start the server, we can actually go to the ingest docs under local development, we can go to ingest dev server. And from here, we can simply copy this command. So let's run this command. Let's install the ingest CLI. And
[11:58] now we can access ingest from this URL. Cool. So that's really it. We can go to apps where we can see all the functions that's available in our application. So we've actually got two we've got generate plushie and generate plushie failure. And on the runs we can see all the completed and in progress runs. Of course, at the moment, we don't have any. So then let's go back to our app. I'm just going to refresh, let's go to generate, then let's generate the splashy. It says the
[12:29] execution has started. And if we go to ingest, we can indeed see our current run. So we can see all the individual steps like uploading the original image, analyzing the image. And then we can also see that this finalization step actually failed with an error. So let's actually go ahead and cancel the step. And in the logs, we get some message saying output too large. So I'm actually going to clear my cloud code conversation. And for context, I'm going to pull in our
[13:01] spec folder. And it's a alright, so we just implemented ingest into our application. I tried to generate an image and I can see the run in ingest. But unfortunately, one of the steps failed. And this is the error message that I can see in the actual state. So I'll just copy this, paste it into the chat. And actually, we can go to the ingest session as well and copy all of this stuff with the error messages. And it's a please use online sources, as well
[13:32] as the context seven MCP server to assist you in troubleshooting this issue. Think hard. Okay, so the problem was we were trying to send the original image to the ingest payloads themselves. And I do have a file size limit. So Claude made a simple change to rather reference the file in blob storage instead, which it should have done from the beginning. So let's try to run this again. And let's switch over to ingest. We can see our process is running. And look at that this time the execution phase
[14:02] completed. So everything seems green. If we go back to our app, we can see the plushie was generated. Awesome. And if we go to gallery, we can see the plushie that we just generated. Now because that previous execution failed, this one still says processing, although it was canceled in ingest. So I'll ask Claude code to also update the status in our app if we end up canceling that run. Thank you. This solution worked. I noticed though that although the previous run failed for that generation,
[14:34] when I go to gallery, that image is still showing processing. So the run is canceled in ingest, but it's showing processing in our application. Is there a way to set the status as failed in our application, or to sync it with the ingest status somehow? All right, so while Claude is working on this, I'm just manually going to delete this entry. There was any new changes that we implemented now won't fix this specific entry. So in the database, we can go to plushie generations,
[15:05] and it's select this one in processing state and whack it. All right, cool. All right, so let's test this again. I just fired off a new generation, and I can see the generation completed in ingest, and we can see our image in the gallery. Cool. So another benefit of using something like ingest now is users can start as many generations as they want. So while that one is running, let's immediately send another one. And in the gallery, we've had two images generated at the same time. And we can see both jobs
[15:37] running in ingest. So this one completed, and that one's still running. And back in ingest, we'll just wait for this one to complete. All right, now all of these runs are complete. And there we go. Now we can see all of our images in the app. Of course, we can keep chatting to Claude Gout to add things like retries and concurrency. But as an MVP, I think this is good enough to deploy to production. Let's actually do that now. I'm going to run the command check bold. This will simply check our solution for any syntax errors. Again, this is a custom command that is included with
[16:09] this free border plate. Right, we don't have any bold errors. So the next thing we have to do is add all of these changes to a commit. Now we can simply create the commit ourselves or call the checkpoint command. This will simply add all of these files to a commit with a detailed comment. And finally, let's click on sync changes. This will push our changes to production. So if we switch over to Vercel, where our project is deployed, you'll see that the project is currently building. Now while the project is being deployed to production,
[16:40] let's set up ingest cloud. Now as I mentioned at the start of the video, ingest is open source. So you can definitely self host it yourself. But for an MVP, I'm simply going to use their cloud platform. Their hobby plan is free and includes up to 5000 executions, which is more than enough for this MVP. So let's sign up. And I'm going to create a new organization. Let's call this plushify AI. Let's create the org. Then let's go to apps. Then in the top right,
[17:12] click on sync new app. So I'll simply go to sync manually. And this will give us this ingest signing key. If you remember, in the implementation plan, Claude told us that for production, we have to create two environment variables, the ingest event key and the ingest signing key. So what we'll do is on our Vercel project, let's go to settings, let's go to environment variables, and let's add our ingest signing key. Then let's copy this key and then add it to this value field
[17:43] and press save. I'll actually select redeploy and redeploy. All right, so for the app URL, we have to provide the URL to our production app with slash API and slash ingest. So of course, we'll simply copy our domain, then we'll add slash API slash ingest. Then let's click on sync app. All right, so it looks like everything was successful. We can see our generate plushy function and all of these different functions. So if we go to runs, we don't
[18:16] have anything at the moment. So let's actually open up the production version of our app and try this out. All right, I actually don't have any credits in production and we haven't implemented the payment functionality yet. So for the time being, what I'm going to do is go to my plushify database in production. Let's open this in neon, let's go to our tables and I'll change the credits to 20 credits for my user profile. Right, so if we refresh the app, we now have 20 credits.
[18:48] Okay, let's try this. So I'll upload this image. Let's generate this plushy and we get this error saying that we couldn't send this event because we couldn't find the event key. And that is because we forgot to add this ingest event key. Silly me, this was actually mentioned in the documentation as well. Let's go back to our project. Let's go to settings environment variables. Let's add ingest event key and to get the value of the key, what we can do is go to ingest. So under production,
[19:19] click on this little key icon, then click on event keys. Then let's create a new key. I'll call this production. Let's copy the key and I'll add it to this value field and click on save. And then let's click on redeploy. Let's grab this image and let's try to generate. So if we go back to ingest, we can see our new run. How cool is that? All right, so my execution initially failed and really all I had to do was go to any
[19:50] of these failed steps and I copy this error and I pass it to Claude code. And it turned out the issue was with my open router API key. I actually removed my key from production after recording that initial video. So all I did was I created a new key, I updated the environment variables and I then clicked on rerun to restart this process. And it just completed now so everything looks okay. I can see the generation in my gallery. How cool is that? Now we have a fantastic solution
[20:21] for seeing all these executions running in the background and we can then restart them or view exactly what went wrong if any of these steps had to fail. Did you find this video useful? Let me know down in the comments. Also remember to leave a like and subscribe to my channel. I'll see you in the next one.