Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Bigger context windows don't fix agent failures when tools return large data — the model forgets the middle of the window due to the attention U-curve. The real solution is context engineering: externalizing large data to memory pointers, selectively retrieving only relevant information, compressing via summarization, and isolating context across agents. The Strands framework (open-source, AWS-maintained) implements these strategies with sliding window, summarization, and combined conversation managers, plus techniques like limited tool invocations and async handles for MCP tools.
Key points
Strands framework offers sliding window, summarization, and combined conversation management strategies.
Memory pointers store large tool outputs externally and reference them by ID in the context window.
Limiting tool invocation counts prevents infinite loops where the agent repeatedly calls the same tool.
Async handles for MCP tools avoid timeout failures by saving results for later retrieval.
Context stuffing and native summarization can degrade agent performance by losing needed information.
Tools mentioned
Techniques
- context engineering
- externalize
- select
- compress
- isolate
- sliding window
- summarization
- memory pointer
- invocation state sharing
- limited tool count
- async handle
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
Hi. Well, first I have to say I am Elizabeth. I am not Morgan. But the guy for the agenda they never change agenda. So we are almost the same but with different colors hair. Okay. So it's
it's going to be similar. So today I going to talk about the infinite content window is a meet. Yes. Yes is because right now edge system breaks when tools returns a large amount of data. Context
window overflow and the engine start to hallucinate and start the recar is degrad. So right now context engineering help us to uh give us a better architecture for this agent application.
So we have some techniques that allows us to improve the context the context window for this agentics application. So imagine that you have a agent that is responsible to uh supervise some
application. So this agent they have to retrieve log logs for that application. So every time this agent is doing the retrieve for the logs is putting data for the content window and every time
this content window is going to grow. It's going to grow every time because when we have a one session and each session is going to retrieve the last session because it's going to remember
everything. That's the way that agents remember and they use the content window for that. In the past about probably one year ago before this agentic era we saw that just add more token that will fix
it. Yeah, there's more bigger content window that is going to fix it. And today we know that is not true because we have the uh attention you curve. What is this? We have a lot of data, but if
that data is super super big, the engine is going to forget the meter in the in the in the curve. It's only going to remember the first part and the last part that we sent to that content
window. And I'm talking here about context engineering. But I can move. But content engineer is giving the model the information it needs when it needed. Sometimes the agent doesn't need all the
information inside that content window. Sometimes the content window is poison the engine because you can have a problem injection inside your content window and the content engineer help us
to incorout the context window for optimal outcomes and token consumption. So today I going to show you some techniques that can help you to improve the content window. And these techniques
uh I manage for a strategy. We have we have the externalize because move this that this big data to a persistent storage. You can create a memory pointer to storage that data. I going to share
the deck at the end. So you need to you don't need to have to. Please pay me attention and because I going to share you all the information that I going to talk share. And the second one is select
you can retrieve only what is relevant. We have compress that reduce token because you can summarize and you can compact that information. And the last one is isolate because you can separate
context across agents. and let's start. So the starting point conversation manager this strategy yeah we have some strategy that are a little complicated but we are going to start for the easy
one that we can use inside the framework. So here in this strategy we are going to use a strand engine is a framework like we are a we maintain is completely open source yes it's free and
is a yeah it's open source free hence it's completely model and so here with strands you can use three different strategy inside the framework so we have sliding window that allows you to keep
only the most recent messenger we have summarization inside the framework that allows you to summarize all their messenger and keep present one and the last one is a combination of two we have
that you can combine summarize of older messenger with resent one so first let's see what is a strand a strand is this so you only need one line of code to create the agent loop so this line here create
all the agent loop behind the scene. You need to create the nose, the graph, nothing because the strands can do that for you and you have the system prom and the tools. And yes, this is simple
because it's using Amazon better behind the scene. But if you want to use Lamba uh Gemini and tropics, you only need to add one line because you can do that. You import the the library for OpenAI
for example and then you add the model line and that's it. So let's return to our strategy. If you want to use summarization conversation manager that's the only line that you that you
can add for this agent loop. So you have the summary radio that is in this case is 50% and you want to preserve preserve the last four messages. So every time that you invoke this agent in this
session, this session is going to manage your content window in this way. So it's going to keep the four last messages and it's going to summarize the other 50%. And yes, this is amazing. But what
happen when you have a lot of memory and you don't want to lose you don't want to lose all the memory inside that content window. So you can use well always we have shortened memory because shortterm
memory is when you have the recent conversation history. So you can use this uh application in the shortterm memory. But what happen when you want to return and uh you for
you uh you have a new session for this agent and you don't have short-term memory. You can have long-term memory and you can create a vector database for that long-term memory. You can have
different types of memory inside that vector database with different prompts that can understand your conversation and say whatever you want. And the third one is relation memory. You can use
entity graph. You can go to the uh to the guy for neo forj that they can explain you more about that. You can use traversal baset or how things connect. So you can understand you can use a
combination of three of those. You can have long short-term memory for the me the session that you have. Then you can retrieve the memory that is in the past the long longterm memory and if you have
some relation thing that you want to keep in a different way you can add for that a graph memory as well. Let's continue with the other one. So what happened with this application
where a lot of logs that I told you you don't want to put that locks inside your content window every time that you invoke the agentic application that agent you can save that inside a memory
pointer. So what how this work? You have the the tools. The tool returns you and a bunch of uh logs. You can use another tools that is going to capable to took that logs and save that logs in a
storage and you are going to retrieve an ID for that storage and for that using a strands you can have that using the agent state. In the agent state you are going to have all the information for
the tools. Then you can use another tool that is going to capable to understand that logs and it's going to send um it's going to send to the agent hey the logs okay you don't need the logs to
understand that but what happen is you need the logs you can go say hey this is my ID come on retrieve the logs because I needed to know so this is something that you can build in your architecture
and it's something like this you can have two different tools that's the way that we create tools inside the trans only with the tool decorator. So we have fetch application logs and if you can
see there you have the tool context agent step state. So we retrieve that information and we save that in a memory pointer. We create a memory pointer ID that is going to preserve inside our
content window. But what happened is your application is super complicated and the tools is not enough. You can use ah if I forget that you can use a multi- aenting application. So you can create a
a trans swam with multi- aents but you share the context window between agents is like add garbage to the other agents because no all the agents need the information between them. So you can
share the information between the invocation state pointer. So you can create an ID between that swam agent swam and you can share the memory pointer between the the invocation state
between that aents and that's the way that we create the swamp in a strand agent and oh my god 9 minutes I going to be fast and the next one is where
the next one is what happened when the engine uh want to stop never in sometimes we have this engine that stay in the loop like forever invoking the same tool the same tool again and it's
never ending so you can stop that with a minimum amount on invocation inside the loop so what is the problem I said the problem to sell more soul may be available so what does happen when you
not when you you don't have a clear response when you evolve the tool So you can avoid that with the tool way with the clear response and with amounts of invocation for the tool and you do like
this. So you have a limited tool count. So you set max tool counts for this example is my tools search fly. You can you only can invoke that tool three times and the other tool. Yeah. Three
time as well. So the agent never is going to do to do this uh internal loop. It's only going to invoke that tools three times. And the last one, yes, I almost there.
The last one is uh is not something that help you to the context window. is something that you can use to help your content window because sometimes we are invoking IMCP tools or IMCP or we have
external API that never answer you at the time that you need it. Sometimes for example we have EP gateway AWS EP gateway it have a a maximum time that is going to receive the the answer. So if
you don't have that time, it's going to uh like stop your application because the delay. So you can avoid that using a a synchron. So when you use synchronous, your engine
is going to wait forever for the answer for your NCP. But if you have an a sync handle, you can invoke your NCP, your external API. So then you can save that and you can retrieve the answer in the
next invocation or when you need it. Yes, this is going to depend of the case that you are building and you can do that like this create MCP this is a function
that use fast API and with fast API you only need to create the tools. So I going to have two different tools. One is the star lob and another one is check job status and I did a lot of test using
this uh a sync handle and the different is a lot. For example, if you have the different um the well the agent never stop to answer me. So because I wait then he can retrieve the information and
to close this session six minutes we have the antipartis so why uh you have to avoid you have to avoid contact stuffing so you need you don't need to include everything and the kitchen you
know you don't have to include everything the content window because the engine doesn't need all the information there you can only share what a need to create the answer that
your user or you're asking for. You can have uh native summarization. Sometimes when you do native summarization, the agent uh can uh you can uh forget what you need. If you're doing whatever
superization what you may for probably in that summarization, you are not going to have everything that the engine is needed to answer you. and context population. Again, don't pass everything
to the agent. The agent doesn't need all that information. And what is a good scene here? So tool returns large data, logs, database, alert information, externalize,
select and use a memory pointer. Many engines need the same large data. You can share invocation between stage and agent loops or tool hands. You can have clearest st you know give a clear
answer for the tool and limited to counts and a thin handle. Thank you. And of course if you want to learn more about stringent we have a huge boot
there. We have a lot of swag. We have a little Legos for Amazon that you can have and see you there. We have the specialist and I am Elizabeth No Morgan remember.
[laughter] Thank you so much. >> [music]