Build Motion Control Interfaces with Multimodal LLMs @JavaScriptConferences
Build Motion Control Interfaces with Multimodal LLMs  @JavaScriptConferences
Uploaded July 2025 | Updated November 2026, 3 hours ago
Talk: Charlie Gerard - "Motion Control With Multimodal LLMs", JSNation 2025

What if you could use multimodal LLMs to interact with websites or IoT devices using motion control?

As advancements in multimodal AI offer new opportunities to push the boundaries of what can be done with this technology, I started wondering how it could be leveraged from the perspective of human-computer interaction.

In this talk, I will take you through my research experimenting with building motion-controlled prototypes using LLMs in JavaScript.

Watch the full talk with Q&A on GitNation website: gitnation.com/contents/motion-control-with-multimodal-llms

Follow the link to watch the full version of all the conference talks, QnA’s with speakers and hands-on workshop recordings β†’ gitnation.com/events/jsnation-2025

Timestamps:

00:00 Introduction to Motion Control with Multimodal AI
01:24 Exploring TensorFlow.js Models for Motion Control
02:47 Enhancing User Interactions with Gesture Control
04:45 Redefining Coding Interfaces with Minimal Inputs
06:58 Leveraging LLMs for Gesture-Based Interaction
08:55 Investigating LLMs for Motion Control Experiences
10:18 Exploring Gemiini for Gesture Recognition
12:34 Webcam-Based Gesture Recognition with Gemini
14:28 Controlling Light States with Gemini Functions
15:46 Interacting with Gemini and WiFi Light Bulbs
17:44 Enabling Multi-Light Control with ID and Position
19:05 Custom Light Names and Gestural Control Logic
20:13 Live API Functions and Directional Light Control
22:57 Integrating TensorFlow.js for Color Control
24:08 Color Changes with Hand Gestures
26:07 Custom Gesture Databases and Autonomous Interfaces
28:28 Personalized Interfaces and Motion Control

πŸ”₯ Want to hear talks like this live? Join us at JSNation US 2025 in New York: jsnation.us/?utm_source=youtube&utm_medium=charliegerard

JSNation website: jsnation.com
JSNation on X: https://x.com/thejsnation
JSNation on LinkedIn: linkedin.com/showcase/jsnation-the-main-js-conference
JSNation on Bluesky: https://bsky.app/profile/jsnation.gitnation.org
JSNation on Instagram: instagram.com/thejsnation
GitNation on TikTok: tiktok.com/@gitnationorg

#JSNation #MultimodalLLMs #GitNation #JSNation2025 #MotionControl #TechTalks
Build Motion Control Interfaces with Multimodal LLMsBring the Magic of 3D to Your Vue Applications With TresJSJSNation US 2026: Hardcore JS Conference in NYC & Online is Open! πŸš€Can we just skip the review? πŸ™ˆ #shortsThink AI skills will get you hired? The answers split #shortsHow to Scale AI Without Breaking the Bank: Typeform’s StrategySenior engineers biggest mistakes (5 honest stories) #shortsMCP Apps: The Spec Behind Imagine, ChatGPT Apps and Shopify | Liad Yosef & Ido Salomon #shortsJoin our convo with Alex Russell, Chrome’s first Web Standards Tech Lead & Microsoft Partner PM πŸ”₯We built a P2P app with no servers. 1M users didnt miss them | David Mark Clements, TetherAuth Under Attack: Catching JavaScript Auth Failures in Production Fast #shortsWhy Duolingo and Shopify are Wrong About AI Strategy | Gregor Ojstersek
JavaScript Conferences by GitNation |

Build Motion Control Interfaces with Multimodal LLMs

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER