Uploaded June 2023 | Updated September 2026, 3 weeks ago
*warning* The video has long-loading times cut out. There are lots of ways to optimize it, plus I suspect the models will get faster, so I don't mind showing what the experience is likely to approach in several month.
AI Agents are quite interesting right now (HuggingGPT, etc.), so I figured I’d take a crack at adapting some of the techniques and see how far it could really go.
The mocked API calls are each expecting a certain data structure to work, so this could indeed be hooked up to Twilio, Gmail, Calendar (would be great with OneGraph!) to give it actual agency.
There are lots of learnings around how to guide GPT to give much more reliable answer formats, to encourage the correct tool/action for a task, etc. Lots of low-hanging fruit to improve it as well, I think.
Then there are general questions around how to structure communication between the tools, e.g. such freeform questions as, “Schedule a meeting with Tom whenever he’s available mid-next week, but not on Thursday afternoon” == What does the resulting data structure look like for a Calendar API to actually extract availability information given this? Fun questions!
Still, it’s surprising to see how far it can go already. And if it were using a GraphQL API I suspect you could automate much/all of the structure information you send it, given the introspectable schema.
*warning* The video has long-loading times cut out. There are lots of ways to optimize it, plus I suspect the models will get faster, so I don't mind showing what the experience is likely to approach in several month.
AI Agents are quite interesting right now (HuggingGPT, etc.), so I figured I’d take a crack at adapting some of the techniques and see how far it could really go.
The mocked API calls are each expecting a certain data structure to work, so this could indeed be hooked up to Twilio, Gmail, Calendar (would be great with OneGraph!) to give it actual agency.
There are lots of learnings around how to guide GPT to give much more reliable answer formats, to encourage the correct tool/action for a task, etc. Lots of low-hanging fruit to improve it as well, I think.
Then there are general questions around how to structure communication between the tools, e.g. such freeform questions as, “Schedule a meeting with Tom whenever he’s available mid-next week, but not on Thursday afternoon” == What does the resulting data structure look like for a Calendar API to actually extract availability information given this? Fun questions!
Still, it’s surprising to see how far it can go already. And if it were using a GraphQL API I suspect you could automate much/all of the structure information you send it, given the introspectable schema.
