Uploaded November 2022 | Updated September 2026, 3 weeks ago
Here we are using websockets to use stable diffusion live while in an AR scene. This ends up being a little challenging because camera access is limited while in webxr.
Timestamps
0:00 Intro
1:05 Steampunk
1:59 Summer and Gothic
2:50 Code Discussion
4:13 Trying yourself
Code*:
stable diffusion websocket script (this code is a little buggy):
github.com/graemeniedermayer/ArExperiments/blob/main/python/websocket-client/stableDiffusionClient.py
webxr JavaScript:
github.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/stableClient.js
django channel routing
github.com/graemeniedermayer/ArExperiments/tree/main/python/django/stableDiffusion
* There are some very big issues with this code notably sending images through json and the websocket-client code is questionable.
Here we are using websockets to use stable diffusion live while in an AR scene. This ends up being a little challenging because camera access is limited while in webxr.
Timestamps
0:00 Intro
1:05 Steampunk
1:59 Summer and Gothic
2:50 Code Discussion
4:13 Trying yourself
Code*:
stable diffusion websocket script (this code is a little buggy):
github.com/graemeniedermayer/ArExperiments/blob/main/python/websocket-client/stableDiffusionClient.py
webxr JavaScript:
github.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/stableClient.js
django channel routing
github.com/graemeniedermayer/ArExperiments/tree/main/python/django/stableDiffusion
* There are some very big issues with this code notably sending images through json and the websocket-client code is questionable.



![Re-texturing Photogrammetry with Stable Diffusion
Some experiments using Stable Diffusion to re-texture photogrammetry models. This is very much a work-in-progress. I wasnt using up-scaling or image masks which could improve results. Theres so many possibilities!
Timestamps
0:00 Intro
0:52 Image textures examples
1:34 The process
2:37 Projected into augmented reality
3:34 Light in Cycles
3:55 Useful tutorial
some of the prompts I used [universe, vampire, cyberpunk, steampunk,french revolution,gothic, samurai, dragon, spring, autumn, rusted]
Super helpful remeshing tutorial:
https://www.youtube.com/watch?v=I-lH2_Ca3Dw
Stable Diffusion
https://github.com/CompVis/stable-diffusion
Meshroom
https://github.com/alicevision/Meshroom
Some other exciting videos I saw this week.
2minutepaper just dropped a video about an amazing extension to stable diffusion thats super relevant to retexturing
https://www.youtube.com/watch?v=XW_nO2NMH_g
Single image of person to 3d model of person:
https://www.youtube.com/watch?v=GK1WdnULYMQ
Inpainting textures in blender:
https://www.youtube.com/watch?v=2rA4Ny-QQfg Re-texturing Photogrammetry with Stable Diffusion](https://i.ytimg.com/vi/yeOuCJ6qCsU/mqdefault.jpg)


