Uploaded July 2026 | Updated September 2026, 3 weeks ago
The next step after a model ships is teaching it to keep learning on the job, and Raymond Feng lays out how Applied Compute trains custom models with reinforcement learning that plug into whatever harness an enterprise already runs. The setup is an orchestrator that fans interactions out to inference engines, collects the graded rollouts, and feeds a training engine that updates the weights, the same GRPO style loop used for RL today, but pointed at real multi turn, long horizon work rather than toy question and answer pairs. The promise is a model you deploy once that adapts to a specific company's tasks.
The hard parts are all about the environment. Feng is candid about reward hacking, where a model learns to time out a tool or exploit a scoring gap instead of doing the task, and about the trouble of faithfully replicating a production environment so training reflects reality. He walks through why replaying real customer interactions is tempting but breaks on non replayability and off policy data, and where automated data pipelines and self evaluation might take this. The vision at the end is a model that learns from every interaction it has, treating each nook and cranny of the job as new training signal.
Speaker info:
- https://x.com/raymondmfeng
Timestamps:
0:00 - Learning on the job
0:39 - Custom models inside your harness
2:37 - Deploy once and adapt
2:49 - The RL training loop
4:40 - Toward longer horizon tasks
6:48 - Reward hacking in practice
9:06 - Replicating production environments
9:45 - Why replaying real traffic is hard
11:57 - Non-replayability and off-policy data
13:41 - Automated data pipelines
15:24 - A model that learns every interaction
The next step after a model ships is teaching it to keep learning on the job, and Raymond Feng lays out how Applied Compute trains custom models with reinforcement learning that plug into whatever harness an enterprise already runs. The setup is an orchestrator that fans interactions out to inference engines, collects the graded rollouts, and feeds a training engine that updates the weights, the same GRPO style loop used for RL today, but pointed at real multi turn, long horizon work rather than toy question and answer pairs. The promise is a model you deploy once that adapts to a specific company's tasks.
The hard parts are all about the environment. Feng is candid about reward hacking, where a model learns to time out a tool or exploit a scoring gap instead of doing the task, and about the trouble of faithfully replicating a production environment so training reflects reality. He walks through why replaying real customer interactions is tempting but breaks on non replayability and off policy data, and where automated data pipelines and self evaluation might take this. The vision at the end is a model that learns from every interaction it has, treating each nook and cranny of the job as new training signal.
Speaker info:
- https://x.com/raymondmfeng
Timestamps:
0:00 - Learning on the job
0:39 - Custom models inside your harness
2:37 - Deploy once and adapt
2:49 - The RL training loop
4:40 - Toward longer horizon tasks
6:48 - Reward hacking in practice
9:06 - Replicating production environments
9:45 - Why replaying real traffic is hard
11:57 - Non-replayability and off-policy data
13:41 - Automated data pipelines
15:24 - A model that learns every interaction
