Uploaded June 2026 | Updated November 2026, 1 day ago
Scaling AI agents from prototype to production is an infrastructure problem.
In this Observe 2026 session, Robert Nishihara, CEO and cofounder of Anyscale, explains how Ray helps teams scale compute-intensive AI workloads, including reinforcement learning, inference, multimodal data pipelines, and custom model development.
Robert covers why reinforcement learning is growing with agents, reasoning models, and custom models; how RL systems combine training, inference, simulations, environments, and agent loops; and why production workloads require distributed Python, GPU scheduling, weight synchronization, and GPU-to-GPU data movement.
You’ll learn:
- Why reinforcement learning is harder to scale than standard model training
- How Ray’s actor API coordinates trainers, generators, and distributed workers
- How teams sync model weights between training and inference
- Why GPU placement, spare capacity, and cluster reliability matter
- Where Ray fits in the AI infrastructure stack for production AI systems
Speaker: Robert Nishihara, CEO & cofounder, Anyscale
Recorded at Arize Observe 2026.
Chapters:
0:00 AI infrastructure for RL and agents
1:20 The origin of Ray at UC Berkeley
2:55 Why Ray adoption exploded
4:10 Real-world Ray workloads at scale
6:15 Why reinforcement learning is exploding
7:58 Ray in frontier model creation
9:15 Implementing RL in Python with Ray
10:10 Trainers, generators, and weight sync
11:18 Ray actors and distributed processes
13:21 The RL training loop
15:13 GPU-to-GPU transfer with RDMA
15:55 Why RL is harder to scale
16:15 Training, inference, environments, and agent loops
19:19 Driver scripts and process placement
20:26 Weight synchronization at massive scale
🔗 Try Arize AX & Phoenix OSS: arize.com
🔔 Subscribe for weekly content on LLMs, agents, and evaluation:
youtube.com/@arizeai?sub_confirmation=1
#Ray #ReinforcementLearning #AIAgents
Scaling AI agents from prototype to production is an infrastructure problem.
In this Observe 2026 session, Robert Nishihara, CEO and cofounder of Anyscale, explains how Ray helps teams scale compute-intensive AI workloads, including reinforcement learning, inference, multimodal data pipelines, and custom model development.
Robert covers why reinforcement learning is growing with agents, reasoning models, and custom models; how RL systems combine training, inference, simulations, environments, and agent loops; and why production workloads require distributed Python, GPU scheduling, weight synchronization, and GPU-to-GPU data movement.
You’ll learn:
- Why reinforcement learning is harder to scale than standard model training
- How Ray’s actor API coordinates trainers, generators, and distributed workers
- How teams sync model weights between training and inference
- Why GPU placement, spare capacity, and cluster reliability matter
- Where Ray fits in the AI infrastructure stack for production AI systems
Speaker: Robert Nishihara, CEO & cofounder, Anyscale
Recorded at Arize Observe 2026.
Chapters:
0:00 AI infrastructure for RL and agents
1:20 The origin of Ray at UC Berkeley
2:55 Why Ray adoption exploded
4:10 Real-world Ray workloads at scale
6:15 Why reinforcement learning is exploding
7:58 Ray in frontier model creation
9:15 Implementing RL in Python with Ray
10:10 Trainers, generators, and weight sync
11:18 Ray actors and distributed processes
13:21 The RL training loop
15:13 GPU-to-GPU transfer with RDMA
15:55 Why RL is harder to scale
16:15 Training, inference, environments, and agent loops
19:19 Driver scripts and process placement
20:26 Weight synchronization at massive scale
🔗 Try Arize AX & Phoenix OSS: arize.com
🔔 Subscribe for weekly content on LLMs, agents, and evaluation:
youtube.com/@arizeai?sub_confirmation=1
#Ray #ReinforcementLearning #AIAgents





![NVIDIAs Peter Belcak Explains Why SLMs are the Future of Agentic AI
In our most recent AI research paper community reading, we had the privilege of hosting Peter Belcak – an AI Researcher working on the reliability and efficiency of agentic systems at NVIDIA – who walked us through his new paper making the rounds in AI circles titled “Small Language Models are the Future of Agentic AI.”
The paper posits that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. The authors’ argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. The authors further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. They discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm.
Learn more about the paper and get a TL;DR: https://arize.com/blog/nvidias-small-language-models-are-the-future-of-agentic-ai-paper/
Ask questions about the research: agents [at] nvidia.com NVIDIAs Peter Belcak Explains Why SLMs are the Future of Agentic AI](https://i.ytimg.com/vi/9xgRTznP21E/mqdefault.jpg)




