This Fixes Claude's Biggest Problem (Karpathy's Method)

0:00 Andrea Karpathy, the former head of AI at Tesla, recently joined Anthropic to fix Claude's biggest problem. This is Claude's ability to remember past conversations and not make the same

0:09 mistake twice. So, I decided to dig deeper and see exactly how Carpathy thinks about this issue. And it turns out the solution is actually quite simple and it already exists in Claw.

0:19 It's their feature dreams. And the reason that you've never heard of it and never used it is that it's only for enterprise companies. So, how can you recreate this feature yourself? Well, in

0:27 this video, I'll provide a simple step-by-step overview of how you can create your own version of dreams so Claude never makes the same mistake twice. But before we do that, we need to

0:36 get a deeper understanding of the problem and how Carpathy thinks about it. And then at the end of this video, I'll cover something that the creator of Claude Code said that relates to this

0:44 topic that is extremely misleading and if you listen to him, you'll waste time and money. So, getting to part one, the problem, how Carpathy sees it, and the fix. to restate the issue. One of the

0:54 biggest problems people face when using Claude is that they repeat themselves over and over again. And Claude continues to make the same mistakes. So, here's how Kurpathy thinks about it. And

1:02 if this sounds too complex, don't worry. I'll simplify it right after. I feel like when I'm awake, I'm building up a context window of stuff that's happening during the day, but I feel like when I

1:09 go to sleep, something magical happens where uh I don't actually think that that context window stays around. Um I think there's some process of distillation into weights of my brain.

1:18 We don't have an equivalent of that in large language models. They're always like restarting from scratch where they were. >> Now, in this interview, he mentioned two

1:25 problems. The first one is that Claude often starts from scratch without any context. This is a problem that you're pretty familiar with. But the second problem is he discusses the process of

1:33 taking short-term contextual information and storing it into long-term contextual information that AI won't forget. And he compares his process of taking short-term memory into long-term memory,

1:44 similar to how a human brain works. When we sleep, we dream, and we bring short-term memories into our long-term memories. So, how do you solve these two problems? Well, the first problem,

1:52 Claude, starting from scratch. You've likely already implemented solutions to help with this. The simplest version is having a single longunning chat for a specific task. And this is something

2:01 that I often do. I have a pin chat. One of them's for accounting. So, whenever I'm talking about my finances, I go directly there. This is a simple solution, but it does help. A slightly

2:09 more advanced version is you leverage a claw.md file, which gets loaded at the start of every new conversation to teach key concepts to the system. And we'll cover that more in a little bit. As well

2:19 as this rules feature that you likely haven't heard of, or the most advanced solution, and you may be familiar with Carpathy's own concept of this, which is LM knowledge bases. And this is the

2:27 process of where you ingest files into your local system to help teach it how you operate. Now, this is a concept that I've covered at length on my channel and one video has over 780,000 views. So, if

2:38 you search Andrea Carpathy Claude on YouTube, I'll be one of the first videos there. Or if it's easier, I link that video below and that'll go through the LLM knowledge basis. Now, that's the

2:46 first problem, and we're going to dive into that more in a little bit. But how do we solve the second problem of taking the context that Claude got during the day and then saving it so it's long-term

2:56 memory so it doesn't forget it in the future? Now, looking back at Carpathy's interview, he hinted Claude needed some sort of dream state where it takes your current short-term memory, cleans it up,

3:06 and then distills it into long-term memory. And after doing some research, it turns out that this already actually exists in Claude. Now, I mentioned this earlier, but there is a dream feature.

3:14 And so on screen you can see the documentation for how it works. But in plain English it does three key things. The first is it takes your existing memory plus up to a 100 of your past

3:24 session transcripts, reads all of it and builds a brand new reorganized memory. Then if there are any duplicates or things in your past memory, it'll merge them and clean it all up. Then it'll

3:33 look for any memories that contradict each other and then pick the winner. One thing from their design structure is that you can also steer the dream. They have an instructions field where you can

3:42 tell it what you want it to focus on and what you want to preserve. Okay, so technical stuff aside, how should you actually think about this? Essentially, just looks at what you did, analyzes it,

3:49 and cleans up your clawed memory so it's smarter and fresher the next day. And the reason that you don't know about this and why you haven't tried is that it runs through the API, which costs

3:57 extra on top of whatever you pay. And it's designed for companies running teams of agents, not necessarily for me and you, which are not these massive enterprise companies. Unless you are

4:07 running an enterprise company, in which case, email me. It is linked in my bio. Let's have a conversation. But anyway, so what we're going to do is recreate this Dreams feature oursel so we can

4:16 have the same exact functionality without being an enterprise company. But before we solve that problem, at the end of the day, these systems are only as good as the data you provide them. And

4:25 the best way to get highquality data is through call transcripts, which brings us to today's video sponsor, Granola, which I am absolutely pumped about because this is a tool that I've been

4:34 using for almost a year now and it is a key part of my AI stack. So Granola is an AI notepad that transcribes your meeting straight off your computer's audio so that no bot has to join the

4:43 call. And then during the call, you can jot notes down directly in the interface just like you would normally do on a notepad. The ability to transcribe calls without a bot joining is such a game

4:52 changer and it's what got me to first use it. But all the other features are why I continue to use it every single day. The first is that when I take client calls, I will take notes on

5:01 specific things they said that stood out to me. Then Granola takes that and drafts a summary with those notes in mind. The second, and this is a feature that I didn't know I needed until I used

5:08 it, is their recipes. A recipe is Granilla's version of a skill that allows you to systematize a specific task. And they have pre-built recipes, but my personal favorite is one I

5:16 created called key focus area. It analyzed my transcripts and identifies what parts of a business could most use AI. I then use this as part of my follow-up, and I do this after almost

5:25 every single sales call I have. So, over time, I'm enhancing this recipe. Third feature is that every call becomes something I can actually query on during and after it. So, one thing I'll do is

5:34 when I'm sitting on a call, if a prospect says an industry term that I'm unfamiliar, instead of just pretending I know what they mean, which I used to do, I'll ask Granola in chat and it gives me

5:44 a definition on the fly. And then after the call, I'll use Granola's MCP to connect directly with Clawed Code, which lets me make more informed decisions and identify the real problems my customers

5:54 and clients are facing that I may have never actually realized or addressed. I personally recommend Granola to basically everyone I work with. So, I'm excited to partner with them and offer

6:03 my community 100% off their first month. So, click the first link in the description to get that all set up. You are going to be happy that you did. Part two is enable your memory system. The

6:12 first part of creating a dreamlike state is making sure we're properly saving memories into our system. So, if you're on a recent version of Claude Code, they have a memory feature that is on by

6:22 default. If you want to look at the setting, open your terminal and in whatever folder you're working on, type claude and then /memory and you'll see the configuration details. This feature

6:30 stores memory files on your computer. So, let's actually go and look at these. And the reason that we're doing this is it will help us understand how we should set up our own dream like system to

6:39 complement our existing memories. So, when you look at the memories folder, you'll have a memory. MD file, which is just a file that points to other files. And then on those other files, that's

6:49 where the details of the actual memory sits. So on the left, you can see the memory MD. And this is really just a reference file. And then on the right, you can see an actual memory that's

6:59 associated with it. So the general structure that they have is a parent memory.md file and then subfiles with all the details about what it wants Claude to actually remember. But that's

7:10 just part of your Claude's memory. The next thing is your claw.md file. Think of this memories.mmd file and your clawed memories as things that just happen in your day. And then clawed MD

7:20 is the equivalent of what you decide to write down on paper so you don't forget it. And if you haven't created a claw.md file yet, don't worry. We'll create it in the next section. But the key with

7:29 both this memory and the claw.md file is they get loaded in at the start of every session. So what that means is if you start a prompt and you say, "Hey, this is Austin." It'll also see all of your

7:40 memory context and all of your claw.md context. Simply put, it's like copy and pasting both files and putting it at the start of the prompt that you just sent. On screen, you can see a table from

7:50 Anthropic Docs comparing these two features and how they relate. So, you now have a deeper understanding of how this all works, which is necessary for the next part where we'll create a dream

7:59 state. Part three is to start dreaming. Going back to what Karpathy mentioned early, the beauty of how people sleep is that humans just automatically remember things. It just happens for them, right?

8:10 We've all experienced this. We're all humans. So, what we want to do is create a system that analyzes our history, reviews our memory and claw.md files, and suggests improvements automatically,

8:21 a lot like a human sleeping and dreaming at night. To do this, we're going to create /dream, an orchestration skill, which is a fancy name for a skill that calls multiple skills. Creating this

8:31 actual dream skill is very simple, but first we need to create the three skills that this will reference. The first is session analysis, then improve memory, then send results. So, let's start off

8:41 by creating session analysis. For each of these, I'm going to walk through the prompt that creates these because it's important to know what's going on, but don't worry to take any screenshots or

8:50 anything. You can just use the buildpartner.ai plugin and run /buildpartner colon buildspace dream and we'll walk you through this step by step. And to get that plugin that is

8:59 linked below. So, what this skill does is it looks at your entire conversation history which is has access to because claude code automatically saves your conversation history to local files.

9:10 Currently, the memory feature we were talking about before just decides what's important, but this looks at your entire conversation history. It reads the raw record of what you actually did and

9:19 pulls out patterns your memory missed. This is a gold mine for information about how your system can start learning better. Then, at the end of the skill creation prompt, we also take in an

9:28 argument called dream. By passing in this argument, it will create an overview of the most useful memories, which will then bring into the next step of this process. The reason that I did

9:37 it like this is that you may want to reuse the same skill in other skills in the future. For example, if I want to see where I'm spending my time, I add an argument. And the one that we're going

9:46 to use is dream. So once you create the whole thing, you would write / session analysis space dream. And it tells a skill that you want to analyze it for memories that you want to bring to your

9:54 long-term memory, which will format it in a way that we'll use in the next step of this process. And the reason that I add it as an argument instead of the default is this session analysis skill I

10:04 may want to use for other workflows in the future. For example, if I wanted to see where I'm spending my time, I could add another argument called time analysis. Then I would write / session

10:13 analysis space time analysis and it would look at this information in a different way. This doesn't necessarily have to do with this specific build, but I want everyone who watches this channel

10:22 starting to understand skill creation best practices. So once we set up that skill, we're going to take the output of that and pass it through the slashimprovememory skill. What this does

10:32 is it looks at your memory and your claude MD files as well as the analysis from the previous step. Much like the existing Dreams feature that Enthropic already built, it'll merge any

10:40 duplicates and it'll fix any contradictions that exist in these files. It'll then suggest upgrades to your claw.md files. So it follows the exact format that Enthropic uses for

10:48 their memory files that I covered earlier. This is a way to keep your claw.md file light and all of the details actually live in separate subfiles. Now, one of the problems with

10:57 just having memories and using a claw.md file is that this is a global configuration for your project. But we really want to get more specific where needed. And this brings us to something

11:06 I literally just found out about when I was researching this video, which makes a ton of sense to me. So, there's a concept called rules where you can have specific memories or instructions that

11:15 only load when you're working on a specific part of a project. So, for example, if I have a project called my life and then within that I have an accounting folder, I can have specific

11:24 accounting rules file that only loads in when I'm working on my accounting stuff. Then, let's say I have a wife folder. You could have wife rules in there that are only specific when you're working on

11:34 things with your wife, which is probably a very extensive file. Again, these are local rules you don't want to make global because it'll cloud the contextual window. If you're working for

11:44 things in the white folder, that doesn't necessarily care about the things in the accounting folder. So, zooming out, essentially what we're doing is we're looking at all of our conversation

11:52 history and identifying places to improve the system. And some of those things are obvious, but some of them are not as obvious. So, what we do is we create a file called memory improvement

12:03 overview. And this is where it'll document any of the changes it'll auto approve, and then it'll flag anything that you, as the human has to improve for your system. Think of this like

12:11 Claude identifying ideas that it thinks you should fix, but you're the one who's finally signing off on it. This is the final human sign off that makes sure that the system is going in the

12:21 direction that you want. Then the last skill that's needed is send results. This is personally how I prefer setting up all of my automations. I have a company automations channel within my

12:29 Slack and that's where all of my automations go. I prefer having things all in one place so I know where to look and I get updated when they run. So for you, this message can be wherever you

12:39 want. This could be your telegram, WhatsApp, email. The key here is having an output that's easily accessible and something that you'll review. Here's a prompt that you could fill in based on

12:47 where you want the results to be sent. So, at this point, we created / session analysis slashimprovememory and slash send results skills. Now, we have to take these three skills and package them

12:57 all together into the orchestration skill, which is dream. On screen, you can see the prompt that actually creates this dream skill. And there are three things to call out. The first is I tell

13:05 it to review the output from the last time it ran. This is so that it applies any changes before it runs the next iteration. I then add an argument for mode which has one mode for full

13:14 analysis and another for apply fixes. I'll dive into why a little bit more when we automate this actual skill. And then third, in the skill itself, you'll see that I reference each of the skills

13:23 that we already created. To reiterate why we do this, each of the subs skills I reference can be used elsewhere in your day-to-day session analysis. You could use the data for other things.

13:33 Improve memory. You may want to just run this manually yourself and send results. Other automations you build will use this skill. So you don't need to reinvent the wheel and you can actually

13:43 scale out systems. This is really where my software engineering background comes in. You design systems based on functions or self-contained logic that you can reuse across your entire system.

13:53 That's what we're doing here so that we can set the foundation for a system that can scale. Now when we package this all up and we actually run the skill, you can see the output. It shows the things

14:01 it automatically approved and then it shows me the things that I need to make a call on myself. And at this point, you can very easily run it manually. Just/dreams and then it'll grip it and

14:10 rip it. But how can we actually automate this? You don't need to complicate this. This is actually very straightforward. Go to claw desktop, click code routines, and then local. And then within that

14:19 actual routine, just reference the /dream skill and then set the cadence for whatever you want. Personally, I set it for weekly on Fridays because I'm just too busy the rest of the week. So

14:28 once you hit save, let's say Friday at 8 a.m., it'll run this dream skill every single Friday. You can do it every night. It's really up to you. But there is a problem at this point. The original

14:37 dream skill will create an overview of changes that I then have to go and review manually. So if we just have this run on Friday, it will wait until the next Friday to apply the changes. And

14:47 that means I'm waiting a week with stale data that I know is an issue that is just sitting there not getting fixed. So what I do is on Saturday I have another routine slashdream apply fixes where

14:57 it'll go and just look for this file and apply fixes automatically for me based on the feedback. So Friday we'll run dreams and then Saturday we'll apply fixes. You can do this at whatever

15:06 cadence you want but this is the structure that works for me. Now we're going through a lot of nitty-gritty details there. So let's take a step back for a second. At this point we've

15:13 established our baseline memory system and created a dream state that will automatically improve for us. And over time, it's going to start bringing those short-term memories into long-term

15:22 memory so you stop having to repeat yourself. Which brings us to a statement from the creator of Claude Code, Boris Churnney, that I don't agree with. He tells you essentially to delete

15:30 everything we just created, which I'm going to dive into. But first, as a thank you to everyone that supports to this channel, I give away a Claude Max subscription every video. And this

15:38 video's winner is Neons. They're building a language learning app. Shout out Neons. And shout out everyone who supports this channel. Now, to enter the next giveaway, I'm genuinely curious.

15:47 What does everyone watching this channel do for a living? Let me know in the comments and I'll try and point you in the direction of a video on my channel that's helpful for you. Everyone

15:54 watching, you're the goats. Subscribe if you haven't. And if you have, appreciate you turning on those post notifications. And that brings us to part four. Don't be misled. So Boris Churnney in a recent

16:03 interview, he was asked how he deals with this exact problem that we covered. And his answer went viral. >> Every 6 months, delete your quadm, delete your skills, delete your hooks,

16:14 see what the model does, and it might surprise you. Now, he's telling you to delete everything essentially, and I don't think he's lying, but there's a lot of nuance here that people need to

16:22 understand versus just reading the headline. Boris is using Claude for coding, and Claude's models are actively getting better and better at writing code. That's entirely the truth. And so,

16:31 what he's saying is that if you have guidance in your project that previously helped Claude solve problems, that guidance might no longer be needed. And that makes sense to me, right? The model

16:40 got better, would have gotten that right anyway. But that's not really what we're doing in this video. The majority of what your memory and claude.md and your skills are doing is directing Claude in

16:50 the route that you want it to go. It literally would never know otherwise. For example, for the dream skill, let's say you deleted it. How would the model know what you wanted it to do? How would

17:00 the model know the output file format you wanted to review? What we're doing is we're guiding Claude in the direction we want versus being overly prescriptive in how to do it. Now, there is a lot of

17:10 nuance here, and I may make a whole video on this topic about the dos and don'ts and what Boris is actually saying, which if you are interested in, you can let me know in the comments, but

17:19 Boris's highlevel point is the same end goal that we have. Get the most out of Claude. And in today's video, we covered Claude's biggest problem. It's not that it doesn't have a memory. It's that

17:29 there's no one saying what it should and shouldn't forget. So, if you want to build exactly what I covered here, there is a link in the description where my free plug-in buildpartner.ai AI will

17:38 walk you through setting it up step by step. Now, once you do set it up, the problem that you'll realize is that this just references your prompt history for data. All of your other data lives out

17:49 in the external world, and that's how you take it to the next level. So, to solve this problem, go check out this video where I walk through setting up all those external connections step by

17:57 step so that you can create a self-improving system. I'll see you over there. pace.