Uploaded March 2026 | Updated November 2026, 3 hours ago
Full write-up with architecture diagrams: arize.com/blog/how-to-manage-llm-context-windows-for-ai-agents
Part one (planning): arize.com/blog/how-to-build-planning-into-your-agent
This is part two of our deep dive series on how we built Alyx, our AI engineering agent — this one is on context management.
Once an agent starts executing a plan, it generates a massive amount of data. Traces, spans, tool results, conversation history — all of it has to fit in a finite context window. How you manage what goes in, what gets trimmed, and what gets dropped entirely determines whether your agent actually works.
In this video we walk through the strategies we landed on after a lot of iteration, and the one approach that looked promising but didn't hold up in practice.
What we cover:
— Middle truncation (keep the start + end, drop the middle)
— Memory with retrieval instead of stuffing everything into context
— Deduplicating messages and pruning tool outputs
— Sub-agents to isolate high-volume tasks
Full write-up with architecture diagrams: arize.com/blog/how-to-manage-llm-context-windows-for-ai-agents
Part one (planning): arize.com/blog/how-to-build-planning-into-your-agent
This is part two of our deep dive series on how we built Alyx, our AI engineering agent — this one is on context management.
Once an agent starts executing a plan, it generates a massive amount of data. Traces, spans, tool results, conversation history — all of it has to fit in a finite context window. How you manage what goes in, what gets trimmed, and what gets dropped entirely determines whether your agent actually works.
In this video we walk through the strategies we landed on after a lot of iteration, and the one approach that looked promising but didn't hold up in practice.
What we cover:
— Middle truncation (keep the start + end, drop the middle)
— Memory with retrieval instead of stuffing everything into context
— Deduplicating messages and pruning tool outputs
— Sub-agents to isolate high-volume tasks







![NVIDIAs Peter Belcak Explains Why SLMs are the Future of Agentic AI
In our most recent AI research paper community reading, we had the privilege of hosting Peter Belcak – an AI Researcher working on the reliability and efficiency of agentic systems at NVIDIA – who walked us through his new paper making the rounds in AI circles titled “Small Language Models are the Future of Agentic AI.”
The paper posits that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. The authors’ argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. The authors further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. They discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm.
Learn more about the paper and get a TL;DR: https://arize.com/blog/nvidias-small-language-models-are-the-future-of-agentic-ai-paper/
Ask questions about the research: agents [at] nvidia.com NVIDIAs Peter Belcak Explains Why SLMs are the Future of Agentic AI](https://i.ytimg.com/vi/9xgRTznP21E/mqdefault.jpg)


