One UI for both Quest 3 and Mobile. Grae n 2024-08-30 | Let's make some UI for mobile AR and passthrough AR at the same time!Code (part of a larger project) github.com/graemeniedermayer/live-3d-gen-basic-game/blob/main/core/ui_components/ui_components.jsUI library usedgithub.com/pmndrs/uikitAnimationgithub.com/tweenjs/tween.js
3 Simple Augmented Reality Projects (Quest webxr) Grae n 2024-05-23 | Be careful with sunlight and quest 3.Today we're going to be doing 3 different quest 3 projects.Timeline0:00 Intro0:28 Gaussian Splats3:52 Throwing app5:59 Porting SimonDevProject 1 Gaussian SplatsSources github.com/mkkellogg/GaussianSplats3D (library used slightly modified example)https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/Splat videos sourcesyoutube.com/watch?v=brr8oO37lkQyoutube.com/watch?v=EXalSlCjrU0&t=205syoutube.com/watch?v=8yfBOcM2WvYProject 2 Hand throwinggithub.com/graemeniedermayer/ArExperiments/blob/main/html/quest_throwing.htmlgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/quest_throwing_app.jsProject 3 Porting threejs projects to quest (using simondev)Code (with ww2 planes instead)github.com/graemeniedermayer/ArAce/tree/questOriginal projectyoutube.com/watch?v=XkvH7z4GxHM
Text to Augmented Reality Grae n 2024-04-04 | Codegithub.com/graemeniedermayer/3d-depth-genHere we are building a prompt to 3d app for quest 3. Very rudimentary but exciting for the future!Heavily uses monodepth big thanks to thygate + semjon00Method for creating 3d modelsyoutube.com/watch?v=_6gD_pEis58Academic Sources@misc{ke2023repurposing, title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation}, author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler}, year={2023}, eprint={2312.02145}, archivePrefix={arXiv}, primaryClass={cs.CV}}@misc{doi.org/10.48550/arxiv.2302.12288, doi = {10.48550/ARXIV.2302.12288}, url = {arxiv.org/abs/2302.12288}, author = {Bhat, Shariq Farooq and Birkl, Reiner and Wofk, Diana and Wonka, Peter and Müller, Matthias}, keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth}, publisher = {arXiv}, year = {2023}, copyright = {arXiv.org perpetual, non-exclusive license}}
The Quest 3 to See Magnetic Fields! Grae n 2024-02-28 | We are using Quest 3, websockets, and a phone to see magnetic fields! The idea is to use the magnetometer on the phone to send measurements into a quest 3 with websockets. Image tracking is necessary to match the reference frames.Chapters0:00 intro1:09 reference frame transform1:50 magnetic examples3:37 codeCode here:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/quest3_magnets.jsgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/mobiles_magnets.js(this repo is messy needs restructuring)
Marigold depth is unique Grae n 2024-01-15 | Marigold is a finetunning of stable diffusion for monodepth. This algorithm is an amazingly flexible for synthetic data. This can be used to create high quality 3d models.Marigold github.com/prs-eth/MarigoldMy noise edits to Marigoldgithub.com/graemeniedermayer/Marigold-Inpainting/tree/mainDepthmap marigoldgithub.com/thygate/stable-diffusion-webui-depthmap-script/tree/marigoldAcademic Citation@misc{ke2023repurposing, title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation}, author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler}, year={2023}, eprint={2312.02145}, archivePrefix={arXiv}, primaryClass={cs.CV}}
Gaussian Splats + Augmented Reality (webxr) Grae n 2023-11-27 | Here we are extending JavaScript gaussian splats renderers to webxr. It works rather well on faster phones. With a few more optimisations for fps it should look very realistic.My Forkgithub.com/graemeniedermayer/ar-example-gsplat.jsGaussian splat JavaScript librariesgithub.com/dylanebert/gsplat.js (library used)github.com/antimatter15/splatgithub.com/mkkellogg/GaussianSplats3D (maybe the best for vr)github.com/cvlab-epfl/gaussian-splatting-webGaussian splats repogithub.com/graphdeco-inria/gaussian-splattingDreamgaussian repogithub.com/dreamgaussian/dreamgaussiandoi.org/10.48550/arXiv.2309.16653Amazing open source repos. Big thanks to everyone!
Dreamgaussian (3d gaussian splatting + dream fusion) Grae n 2023-10-26 | Repo being usedgithub.com/dreamgaussian/dreamgaussiandoi.org/10.48550/arXiv.2309.16653Super awesome repo. Big thanks to the authors!This repo/paper combines a large number of ideas including 3d gaussian splatting, 3d generative ai, single image reconstruction, and a number of other numerical wizardry. Neural/Differential rendering are taking off much faster than I expected!Very useful threadgithub.com/dreamgaussian/dreamgaussian/issues/22
AI Tangent Space Normal Maps Grae n 2023-10-19 | Here we are creating tangent space normal maps using ai gen and zoedepth. The goal is to reduced an mono depthmap to a lower resolution while maintaining the 3d visual quality. Tangent normal maps also have the advantage of working with deformable meshes.Code is here (I'm thinking about finding a better place for it):github.com/graemeniedermayer/stableScripts/blob/main/tangent_normalmaps.pyExtra normal map resourcesFreya Holmeryoutube.com/@UC7M-Wz4zK8oikt6ATcoTwBAaVersionOfRealityyoutube.com/@aVersionOfReality
Stylized AI Normal Maps Grae n 2023-09-01 | Today we are using stable diffusion to edit normal maps!TimelineIntro 0:00Manual Stylizing 0:30Gen Normals 1:05Materials 2:173d Normals 2:57Shadows 3:22Original Videoyoutube.com/watch?v=s8N00rjil_4 (youtube version)tiktok.com/@codygindy/video/7264345365950795050?lang=enImpasto Loracivitai.com/models/82582/thick-impasto-painting
Image to 3d, the Han Solo Method Grae n 2023-07-26 | We're exploring a useful workflow for turning images into 3d models. This uses the boolean modifier and I'm really happy with how close it is to full automation.Timeline0:00 Intro0:33 Img tips2:00 Depth tips4:35 Blender6:45 MixamoDepthmapsgithub.com/thygate/stable-diffusion-webui-depthmap-scriptAdd detail loracivitai.com/models/58390Other img-to-3d videosyoutube.com/watch?v=tBLk4roDTCQyoutube.com/watch?v=Wf-OmHyFduoAlso for post processing AI models this videos is really interestyoutube.com/watch?v=-rfNOBGIiykgaussian edge blur script and opencv contour script available heregithub.com/graemeniedermayer/stableScripts/tree/mainIf this was too fast-pace dense let me know.
Drag3d has arrived. Grae n 2023-06-15 | Drag3dgithub.com/ashawkey/Drag3DAcknowledgementsgithub.com/XingangPan/DragGANgithub.com/nv-tlabs/GET3DSuper excited open source repo! You can drag 3d models around and they will be constrained to the original object type.
Optical Illusions for Depth AI Grae n 2023-06-02 | There's a lot of ambiguity that arises from mono depth. Are we talking about physical depth or mental depth. Here we'll explore the edge cases of mono depth such as mirrors, refraction, projections, and rainbows.This was put together usinggithub.com/thygate/stable-diffusion-webui-depthmap-scriptProjectionist / Visuals / Musicianinstagram.com/_fiatlvx/?hl=eninstagram.com/1000joules/?hl=en
Zoe depth is impressive. Grae n 2023-04-26 | Zoe depth is a relatively new monocular depth algorithm. It combines really nicely with other algorithms. Here we are using it with sadtalker to fun animated meshes!ResourcesAcademic PapersZoedepth: arxiv.org/abs/2302.12288github.com/isl-org/ZoeDepthSadtalker: arxiv.org/abs/2211.12194github.com/Winfredy/SadTalkerYoutubeMickmumpitzyoutube.com/@mickmumpitzPrompt muse channelyoutube.com/@promptmuseDepth repo:github.com/thygate/stable-diffusion-webui-depthmap-script
ControlNet to 3d with Nerfs. Grae n 2023-04-06 | Here we are outlining an improved way to get ai images to 3d. It's getting so close to high quality now! Also Dreambooth3d is exciting!timestamps0:00 intro1:29 Google's Dreambooth3d2:45 My pipeline3:38 My results5:18 Emergent 3d?Sourcesacademic-dreambooth3d arxiv.org/abs/2303.13508opensource-mikubull's controlnet extension github.com/Mikubill/sd-webui-controlnet -controlnet repo - github.com/lllyasviel/ControlNet-my scripts github.com/graemeniedermayer/stableScriptsThe models were found on civitai and huggingfaces sites
Inpainting in Augmented Reality Grae n 2023-03-06 | This is an example of AI inpaint within augmented reality (webxr). This is a messy space but also fascinating. If you have any suggestions let me know!Code (very messy): github.com/graemeniedermayer/augmented_reality_SD/blob/main/frontend/js/autoInpainting.js
Setting up an AI Art and Augmented Reality app Grae n 2023-02-09 | This is a quick rundown on github.com/graemeniedermayer/augmented_reality_SD a webxr setup for automatic1111 (stable diffusion).It's a work in progress there's a lot of fixing to do but I'm super excited about the possibilities
Auto1111 API and Augmented Reality Grae n 2023-01-21 | Using the Automatic1111 api with augmented reality. This is getting pretty crazy. I'm definitely going to be expanding on this idea.messy code (hopefully I'll have some cleaner code next week): github.com/graemeniedermayer/ArExperiments/blob/main/javascript/auto1111ar.js
Remaking 3D Maze with all the buzzwords! (AI art, Augmented Reality, AI Depth, and more) Grae n 2023-01-12 | Just some fun with AR, AI, and I guess kafka? A maze built with buzzwords.Demo (Android ARCore only):graemeniedermayer.github.io/ArExperiments/html/maze.htmlCode:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/arMaze.jsMiDaS (for normal maps) :@article {Ranftl2022, author = "Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun", title = "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer", journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence", year = "2022", volume = "44", number = "3"}
Easy 3d Inpainting (with AI art) Grae n 2023-01-06 | This highlights new functionality of the depthmap repo which is just super fun. 3d inpainting is super easy to use with AI art.The amazing depthmap repo github.com/thygate/stable-diffusion-webui-depthmap-scriptAcademic reference@inproceedings{Shih3DP20, author = {Shih, Meng-Li and Su, Shih-Yang and Kopf, Johannes and Huang, Jia-Bin}, title = {3D Photography using Context-aware Layered Depth Inpainting}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2020}}
AI Art Scripting (Clothing Masks, Background Removal, and Automatic1111 api) Grae n 2022-12-28 | I wanted to go over some of the common scripts I've been using for AI art.Timeline0:00 Intro1:15 opencv Contour1:48 mediapipe2:38 rembg3:50 automatic1111 api5:03 numpy image scriptingCode Main repo - github.com/graemeniedermayer/stableScriptsAuto1111 clothing seg extension (work-in-progress) - github.com/graemeniedermayer/clothseg
The Crazy World of Neural Rendering Grae n 2022-12-16 | The field of Neural Rendering has grown enormously over the past 2 years. The field has already contributed to txt-to-3d, single-shot image-to-3d, meshless 3d, and novel view synthesis with much more promise for the future. These methods have proven to be both high quality, flexible, and robust. Let's explore Neural Rendering!Timestamps:0:00 Intro0:56 NeRF 2:00 NGP + mip3603:25 SNeRG + NeRV + dNeRF6:03 Single views7:18 Latent Dif + NeuralRepostorch-ngp - github.com/ashawkey/torch-ngpngp_pl - github.com/kwea123/ngp_plAcademic Papers: I ran out of characters with standard citations. You should be able to google the arXiv citation, if you have trouble find a source let me know! Many of the papers from 2022 may still be in pre-print (non-peer reviewed). [0] arXiv:2111.11426 - nerf review[1] arXiv:2210.00379 - nerf paper[2] arXiv:2201.05989 - ngp paper[3] arXiv:2111.12077 - mip360 paper[4] arXiv:2103.14645 - snerg paper[5] arXiv:2012.03927 - nerv paper[6] arXiv:2011.13961 - dnerf paper[7] arXiv:2204.00928 - sinnerf paper[8] arXiv:2210.08936 - s^3 nerf paper[9] arXiv:2212.03267 - nerdi paper[10] arXiv:1901.05103 - deepSDF paper[11] arXiv:2104.09877 - shadow nerfs paper[12] arXiv:2211.10440 - magic3d paper[13] arXiv:2209.14988 - dreamfusion paperNot used but referred to[14] arXiv:2203.12575 - neuMan paper[15] arXiv:2203.07182 - NeILF paperThanks arXiv for being free!
How to relight any AI art piece Grae n 2022-12-03 | It's crazy this works as well as it does. You can relight ai pictures/animations very easily and you can also bake the shadows so they look better! Let's hope YouTube video compression doesn't destroy the quality too much!This should work for non-ai art. You might also be able to get this workflow too work with just a depth map. I have not tried using depth2img with this. In principle I'd expect it to give better results; however, I have not tested it.Timestamps:0:00 Intro0:20 Workflow1:47 Problems and fixes3:10 Materials?4:40 Other Experimentsnormal map plug-in:github.com/graemeniedermayer/stable-diffusion-webui-normalmap-scriptdepth plug-in (super huge thanks to thygate):github.com/thygate/stable-diffusion-webui-depthmap-scriptmidas:github.com/isl-org/MiDaS
Normal Maps for Stable Diffusion Grae n 2022-12-01 | Okay so it's actually a heavy fork/small modification of thygate's depthmap extension. Great job to thygate! We can make ai normal maps! I made an extension that makes it easier. If anything goes wrong let me know!normal map plug-in:github.com/graemeniedermayer/stable-diffusion-webui-normalmap-scriptdepth plug-in (super huge thanks to thygate):github.com/thygate/stable-diffusion-webui-depthmap-scriptmidas:github.com/isl-org/MiDaS
Walking into AI Art (using monocular depth sensing) Grae n 2022-11-27 | You can project AI art into 3d using monocular depth sensing. It works really well. This has huge implications in lighting, physics, parallax motion, and many many other areas. Today we're combining it with augmented reality so that you can walk inside of ai art.webxr mobile demo (no iphone support) (stable diffusion 2 examples):graemeniedermayer.github.io/ArExperiments/fantasyDepth.htmlcode:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/fantasyDepth.jsdepth plug-in (super huge thanks to thygate):github.com/thygate/stable-diffusion-webui-depthmap-scriptmidas:github.com/isl-org/MiDaS
3d to 3d ai art with stable diffusion and NeRFs. Grae n 2022-11-21 | In this video we are going to explore using NeRFs (neural radiance fields) in conjunction with imgtoimg to create new 3d objects. It's a little rough around the edges but this will improve as video coherence improves!Sorry for the low resolution.timestamps0:00 intro0:28 Lenticular Analogy1:20 Pose Experiment1:50 RCMP to Goth2:30 Truck to Art Supplies3:05 Trying it yourself!4:30 Differences with DreamfusionAcademic Sources (they include code)nerfs - matthewtancik.com/nerfllff - bmild.github.io/llffcolmap - colmap.github.iodream fusion (no code) - dreamfusion3d.github.io Sources:how to make lenticular imagesyoutube.com/watch?v=38lMTSIXrokaugmented thinker's post reddit.com/r/augmentedreality/comments/yls9b6/i_believe_i_nerfd_a_first_lava_lamp_with_motion_isarai marte's video that made me think about layeringyoutube.com/watch?v=JtRDVmqqn7c
Combining the Depth API and Stable Diffusion Grae n 2022-11-09 | Here we are using websockets to use stable diffusion live while in an AR scene. This ends up being a little challenging because camera access is limited while in webxr.Timestamps0:00 Intro1:05 Steampunk1:59 Summer and Gothic2:50 Code Discussion4:13 Trying yourselfCode*:stable diffusion websocket script (this code is a little buggy):github.com/graemeniedermayer/ArExperiments/blob/main/python/websocket-client/stableDiffusionClient.pywebxr JavaScript:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/stableClient.jsdjango channel routinggithub.com/graemeniedermayer/ArExperiments/tree/main/python/django/stableDiffusion* There are some very big issues with this code notably sending images through json and the websocket-client code is questionable.
Combining Stable Diffusion and Particle Systems Grae n 2022-10-30 | For follow alongGoogle Colab:github.com/graemeniedermayer/ArExperiments/blob/main/notebooks/minimal_img2img.ipynbParticle Image used:github.com/graemeniedermayer/ArExperiments/blob/main/data/images/particles.pngWebapp:graemeniedermayer.github.io/ArExperiments/html/particleSystemUploadable.html timestamps0:00 Intro1:00 Follow along google colab2:30 Using these particle system in blender3:37 Using these particle systems in unity4:24 Fun math problem6:20 Directional particle systemJavascript for this project is locatedgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/arParticleSystem.jsgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/arDirectionalParticles.jsgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/particleSystemUploadable.jsToday we are exploring using stable diffusion to create special effects with particles systems. Stable diffusion img2img can help create unique special effect very easily. A lot of this code originated from @SimonDev particle system videos.
AI Word Art and Fonts with Stable Diffusion Grae n 2022-10-19 | This is about using stable diffusion to create word art and fonts. Taking a pre-existing words and img-to-img algorithm it. Other word art / useful related videos I've found are:bitmaps to svgs - youtube.com/watch?v=cNXGnEKz0Rwchannel with lots of word art - youtube.com/channel/UCXJpHQFclSVm_PkPwDQeGQg/videos?view=0&sort=p&flow=gridmore consistence ai videosyoutube.com/watch?v=xtFFKDgyJ7ATimestamps0:00 Intro0:13 Word Art1:00 Animations2:51 Fonts5:20 OutroTo be clear this was using img2img from:github.com/CompVis/stable-diffusionNOT (which is higher in googles algorithm)huggingface.co/spaces/lambdalabs/stable-diffusion-image-variations
Re-texturing Myself with Stable Diffusion, Photogrammetry, and a little PIFuHD Grae n 2022-10-12 | Thanks for all the kind and helpful comments on previous videos!This is very much a work-in-progress. There's still lots of room for improvement. In particular you could probably get better results from using cleaner meshes.It seems like new variants of stable diffusion are being release every few days.Time stamps0:00 Intro0:12 Augmented Reality (UV Maps)2:36 Hands3:00 Feet3:41 Stable Diffusion fixing artefacts4:11 PIFuHD examples4:27 Photogrammetry5:13 More FacesStable Diffusiongithub.com/CompVis/stable-diffusionMeshroomgithub.com/alicevision/MeshroomPIFuHDshunsukesaito.github.io/PIFuHDSome other exciting videos I saw this weekSingle image of person to 3d model of person (I used this for pifuHD):youtube.com/watch?v=GK1WdnULYMQ
Re-texturing Photogrammetry with Stable Diffusion Grae n 2022-10-07 | Some experiments using Stable Diffusion to re-texture photogrammetry models. This is very much a work-in-progress. I wasn't using up-scaling or image masks which could improve results. There's so many possibilities!Timestamps0:00 Intro0:52 Image textures examples1:34 The process2:37 Projected into augmented reality3:34 Light in Cycles3:55 Useful tutorialsome of the prompts I used ['universe', 'vampire', 'cyberpunk', 'steampunk','french revolution'','gothic', 'samurai', 'dragon', 'spring', 'autumn, 'rusted']Super helpful remeshing tutorial:youtube.com/watch?v=I-lH2_Ca3DwStable Diffusiongithub.com/CompVis/stable-diffusionMeshroomgithub.com/alicevision/MeshroomSome other exciting videos I saw this week.2minutepaper just dropped a video about an amazing extension to stable diffusion that's super relevant to retexturingyoutube.com/watch?v=XW_nO2NMH_gSingle image of person to 3d model of person:youtube.com/watch?v=GK1WdnULYMQ Inpainting textures in blender:youtube.com/watch?v=2rA4Ny-QQfg
Using AI art to create game environments. Grae n 2022-10-04 | To be clear a good portion of the backdrop is the real world. Here we are using AI art (from stable diffusion) to create an environment for our character created in the last video. The prompts used were variations of "____ building concept art" and "____ tree concept art" with styles and seeds varied.This project uses a fairly significant amount of code from simondev .Timestamps0:00 Intro1:05 Colliders2:16 Goals2:53 Code Discussion3:50 CodeThe Repo is (Heavy WIP) here:github.com/graemeniedermayer/aiar
Re-texturing with Stable Diffusion Grae n 2022-09-29 | This is about using stable diffusion to re-texturing. Taking a pre-existing texture and img-to-img algorithm it. The x-wing and anime character were from sketchfab I think.Other texture related videos I've found are:youtube.com/watch?v=UvJkQPtr-8syoutube.com/watch?v=FmY5AiempIIyoutube.com/watch?v=Me8-dKhQ0UUTo be clear this was using:github.com/CompVis/stable-diffusionNOT (which is higher in googles algorithm)huggingface.co/spaces/lambdalabs/stable-diffusion-image-variations
Using AI Art to make games (part 1 character creation) Grae n 2022-09-22 | Today we are using Stable Diffusion, an AI algorithm, to create art asset for a game. This video focuses on creating a 2.5d character (3d animations and 2d models). We are using webxr, threejs, and Javascript.Time Stamps:0:00 Intro0:17 Using Stable Diffusion2:02 Image Processing2:23 Blender4:49 Comparing My Art with the AI5:21 Code8:03 Up NextLinksStable Diffiusion web app:huggingface.co/spaces/stabilityai/stable-diffusionStable Diffiusion Google Colab Book:colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/stable_diffusion.ipynbCode (Me also a WIP):github.com/graemeniedermayer/aiar
Jarvis Visualization (AlexNet mapped) Grae n 2022-09-15 | This is a silly video about trying to recreate jarvis in AR using the neural network weights from alexnet. While Jarvis itself is a rather impracticable visualization, I do think there is pragmatic reasons for neural networks to be visualized in augmented reality.Code:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/jarVis.js
Tracking images to make procedural terrain (with Catan as an example) Grae n 2022-09-08 | Using procedural terrain generation and image tracking together to recreate an Augmented reality terrain model of a catan board. These ideas can certain be applied to other board games as well. Credit to @SebastianLague and @simondev758 for teaching me about procedural terrain.Timestamps:Intro 0:00Image Tracking 0:36Delanauy Traingulation (aside) 2:25Image Tracking Second Part 3:10Terrain Generation 3:35Combining Terrain and Tracking 5:04Code 6:39Code is located at (You'll need to take your own pictures):github.com/graemeniedermayer/ArExperiments/blob/main/javascript/catanTerrain.js
Neural Network in Augmented Reality (AlexNet) Grae n 2022-08-30 | A demo is should be available at graemeniedermayer.github.io/ArExperiments/NNVis.htmlThis is a visualization of the wieghts of AlexNet a neural network that won a competition in 2012 that kicked off some of the more recent advancements in artificial intelligence. Many more recent networks are much more complicated but they are still often represented by blocked of weights/parameters.Time Codes0:00 AlexNet1:17 Basic Walkthrough2:30 More Through Walkthrough8:04 Image To Weights9:01 Visualizing with Units or Connections10:06 CodeThere's a lot of sourcesmedium.com/@sdoshi579/convolutional-neural-network-learn-and-apply-3dac9acfe2b6github.com/mikechen66/AlexNet_TensorFlow2.0-2.3 (this was the alexnet version I used)en.wikipedia.org/wiki/AlexNetproceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf (alexnet paper)This visualisation contains all of the 60,954,656 weights of AlexNet. Gosh I need to learn how to do 3d video stabilization.Code is atgithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/arVisNN.js
Shared Realities in webxr Grae n 2022-08-20 | This shows a minimal shared reality setup using image tracking within webxr.Timestamp0:00 Intro0:56 Maths2:09 Example4:47 CodeCodegithub.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/followerWebsocket.js
Image Tracking with Webxr Grae n 2022-08-05 | This is a tutorial in using webxr's image tracking api. We're doing two simple projects: a gravity simulation and an animation tracked to cards. This is a new/experimental api so it might not work on everyone's phone.Timestamps0:00 Intro0:48 Quick Demos2:16 Image Quality3:29 Gravity Simulation5:07 Skeleton Code6:55 Animation Code7:35 Gravity Sim CodeDemo:gravity:graemeniedermayer.github.io/ArExperiments/html/gravityMarker.htmlskeleton:graemeniedermayer.github.io/ArExperiments/html/webxrMarker.htmlCode:gravity markers: github.com/graemeniedermayer/ArExperiments/blob/main/javascript/gravityMarkers.jsskeleton: github.com/graemeniedermayer/ArExperiments/blob/main/javascript/webxrMarker.jsanimation: github.com/graemeniedermayer/ArExperiments/blob/main/javascript/animationMarker
Physics-based Physics Simulations (in webxr) Grae n 2022-07-30 | This video uses websocket and webxr to record a pendulums mostly precise trajectory. The information is then saved and we can do cool things like look at energy conservation.Time Codes:0:00 Intro0:53 Charts2:17 Energy conservation discussion3:56 Code_the code discussed:github.com/graemeniedermayer/ArExperiments/blob/main/javascript/websockets/websocketCharts.js
Websockets and Augmented Reality Grae n 2022-07-26 | This demonstrates using websockets together with augmented reality (webxr). The red box on the right is following the kinematic information for the phone on the left. The two videos are synced manually but because of some frame drops they didn't line up exactly.**Please make sure websockets are secure if you do anything in the wild with this.Time Stamps0:00 Intro2:24 Server setup3:22 Example4:30 JavaScript Client Code6:14 Python Server CodeDjango Channels / Nginx setups:1. digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-18-04#step-8-%E2%80%94-checking-for-the-gunicorn-socket-file2. digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04 3. channels.readthedocs.io/en/stableCode: github.com/graemeniedermayer/ArExperiments/tree/main/javascript/websockets
Post Processing in Augmented Reality Grae n 2022-07-17 | How to use post processing pipeline in webxr. This uses threejs, webxr, and is written in JavaScript.Live DemoBloom:graemeniedermayer.github.io/ArAce/postProcessBloom.htmlAfterImage:graemeniedermayer.github.io/ArAce/postProcessAfterImage.htmlDepth of Field:graemeniedermayer.github.io/ArAce/postProcessDOF.htmlCode is here:github.com/graemeniedermayer/ArAce
Easier Occlusion in webxr Grae n 2022-07-06 | A messier but easier way to do occlusion in webxr. This is a great way to do rough multi-object occlusion.demograemeniedermayer.github.io/ArAce/occlusionShadow.htmlcodegithub.com/graemeniedermayer/ArAce/tree/main/src/occlusionShadow
Shadows and Light Estimation in a WW2 Augmented Reality Game Grae n 2022-06-28 | Demo for chrome+android : graemeniedermayer.github.io/ArAce This video focuses on using depth information to cast shadows and using light estimation. It is primarily webxr, threejs, and javascript.This uses a fair amount of @simondev758 's code.This is a code is available here: github.com/graemeniedermayer/ArAce
Plane Detection in webxr Grae n 2022-06-20 | This is a quick video of using plane detection. The immersive web example was very helpful.Demo: graemeniedermayer.github.io/ArExperiments/html/arPlaneDetection.html Code: github.com/graemeniedermayer/ArExperiments/blob/main/javascript/planeDetect.js
Porting Simondevs Pew Pew game to webxr Grae n 2022-06-18 | This is basically a quick overview of porting of @SimonDev 's game into webxr.simondev original video youtube.com/watch?v=XkvH7z4GxHM code: github.com/graemeniedermayer/PewPewInARsimondev original github: github.com/simondevyoutube/Quick_Game2_StarWarsThing/tree/main/src
