Uploaded October 2026 | Updated November 2026, 11 hours ago
Karan Gathani and the Shiny team at Posit benchmark various LLMs across various measures and identify key strategies to optimize their performance. They take building excellent Shiny apps very seriously, whether you're building them line-by-line or with LLMs. LLM hype is everywhere, but what interests Karan most is moving beyond marketing claims to demonstrate how these models can be scientifically improved for specific use cases. That way, we can assess with confidence whether newly released models are genuinely better or worse.
Karan is a Senior Quality Assurance Engineer at Posit, where he works on Shiny. He has worked with tech startups across the San Francisco Bay Area, helping them ship products efficiently without compromising on quality. In his spare time, he takes part in BioBlitzes around the Bay Area and contributes to this natural history blog.
The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We'd love to have you! It's is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow.
Hosting crew from Posit: Libby Heeren, Isabella Velasquez
Karan Gathani's GitHub: github.com/karangattu
Resources:
Download Positron (IDE that Karan uses): positron.posit.co/download.html
Vision language models are blind: openaccess.thecvf.com/content/ACCV2024/papers/Rahmanzadehgervi_Vision_language_models_are_blind_ACCV_2024_paper.pdf
Open Code: opencode.ai
The vitals package! Writing evals for your LLM product: vitals.tidyverse.org/articles/writing-evals.html
Inspect (for Python): inspect.aisi.org.uk
Karan's blog post on Inspect: posit.co/blog/using-evals-in-python
Built-in Skill for Shiny in Posit Assistant (using bslib!): github.com/posit-dev/skills/blob/main/shiny/shiny-bslib/SKILL.md
Agent Skills overview: agentskills.io/home
Shinygen: karangattu.github.io/shinygen
Karan Gathani and the Shiny team at Posit benchmark various LLMs across various measures and identify key strategies to optimize their performance. They take building excellent Shiny apps very seriously, whether you're building them line-by-line or with LLMs. LLM hype is everywhere, but what interests Karan most is moving beyond marketing claims to demonstrate how these models can be scientifically improved for specific use cases. That way, we can assess with confidence whether newly released models are genuinely better or worse.
Karan is a Senior Quality Assurance Engineer at Posit, where he works on Shiny. He has worked with tech startups across the San Francisco Bay Area, helping them ship products efficiently without compromising on quality. In his spare time, he takes part in BioBlitzes around the Bay Area and contributes to this natural history blog.
The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We'd love to have you! It's is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow.
Hosting crew from Posit: Libby Heeren, Isabella Velasquez
Karan Gathani's GitHub: github.com/karangattu
Resources:
Download Positron (IDE that Karan uses): positron.posit.co/download.html
Vision language models are blind: openaccess.thecvf.com/content/ACCV2024/papers/Rahmanzadehgervi_Vision_language_models_are_blind_ACCV_2024_paper.pdf
Open Code: opencode.ai
The vitals package! Writing evals for your LLM product: vitals.tidyverse.org/articles/writing-evals.html
Inspect (for Python): inspect.aisi.org.uk
Karan's blog post on Inspect: posit.co/blog/using-evals-in-python
Built-in Skill for Shiny in Posit Assistant (using bslib!): github.com/posit-dev/skills/blob/main/shiny/shiny-bslib/SKILL.md
Agent Skills overview: agentskills.io/home
Shinygen: karangattu.github.io/shinygen










