Uploaded July 2026 | Updated September 2026, 1 week ago
Smelter's new side channel hands already-decoded frames - video or audio - straight to an external process, no re-decoding needed.
In this episode of the Smelter Workshop I put it to work three ways: fully local speech-to-text with Whisper, scheduling captions so they land exactly on the speech; car detection on bird's-eye drone footage feeding a WGSL shader that repaints every car live; and a swarm of ghost sprites that chase people around the frame using YOLO detections at 60 fps. All composed and broadcast over WebRTC as one live Smelter stream.
🔗 Smelter: https://smelter.dev
🔗 Live Demo: https://workshop.smelter.dev
Repos:
🔗 Project repo: github.com/smelter-labs/smelter-editor
Smelter's new side channel hands already-decoded frames - video or audio - straight to an external process, no re-decoding needed.
In this episode of the Smelter Workshop I put it to work three ways: fully local speech-to-text with Whisper, scheduling captions so they land exactly on the speech; car detection on bird's-eye drone footage feeding a WGSL shader that repaints every car live; and a swarm of ghost sprites that chase people around the frame using YOLO detections at 60 fps. All composed and broadcast over WebRTC as one live Smelter stream.
🔗 Smelter: https://smelter.dev
🔗 Live Demo: https://workshop.smelter.dev
Repos:
🔗 Project repo: github.com/smelter-labs/smelter-editor
