Mickmumpitz | Generate 3D Sets for your Short Films! @mickmumpitz | Uploaded March 2023 | Updated October 2024, 3 hours ago.
Today we'll be using AI tools to create virtual environments for your film projects!
We cover the entire workflow, from generating the 3D environment, to filming in a low-budget virtual production environment, to compositing your shot with some AI tips and tricks, like re-lighting your shot based on the environment you've created!
Chapters:
00:00 Intro
00:34 What is Virtual Production?
01:42 Generating a 360° image
03:17 Preparing the 3D Model [Depth Maps]
06:22 Filming the scene
08:05 Key out the actors
09:10 Generate Normal Maps
11:15 Putting it all together
14:11 Final Shots
Tools used in this video:
- Skybox Labs by Blockade Labs: skybox.blockadelabs.com
- NVIDIA Canvas: nvidia.com/de-de/studio/canvas
- ControlNet for Stable Diffusion: github.com/lllyasviel/ControlNet
- midjourney: midjourney.com
- High Resolution Depth Maps for Stable Diffusion WebUI: github.com/thygate/stable-diffusion-webui-depthmap-script
- Blender: blender.org
- CamTrackAR: apps.apple.com/us/app/camtrackar/id1502545167
- Reality OBJ to USDZ Converter: apps.apple.com/de/app/reality-obj-usdz-converter/id1603967912
- After Effects: adobe.com/products/aftereffects.html
- Runway.ml: runwayml.com
- High Reslution Normal Maps for Stable Diffusion: github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script
- DaVinciResolve: blackmagicdesign.com/products/davinciresolve
- Reference to image plane: github.com/Pullusb/reference_to_image_plane
Videos Links:
- Avatar Featurette: Performance Capture: youtube.com/watch?v=OJ1JzYPjcj0
- Avatar & Avatar 2 Behind the scenes - How James Cameron Evolved Motion Capture in the Avatar Films: youtu.be/WrKL432F86Q
- The Virtual Production of The Mandalorian Season Two: youtu.be/-gX4N5rDYeQ
- Blender Motion Tracking - Room Transformation! by Ian Hubert: youtu.be/lY8Ol2n4o4A
- Grae n YouTube Channel: youtube.com/@UC_EkMqKOm7Ej_g03Ryv0UZw
- Inpainting in Augmented Reality: youtube.com/watch?v=wJNROIpjveI
- How to Make a Blur Node in Blender!: youtube.com/watch?v=IHmN8V4Yqwk
❤️ Buy me a coffee:
ko-fi.com/mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Today we'll be using AI tools to create virtual environments for your film projects!
We cover the entire workflow, from generating the 3D environment, to filming in a low-budget virtual production environment, to compositing your shot with some AI tips and tricks, like re-lighting your shot based on the environment you've created!
Chapters:
00:00 Intro
00:34 What is Virtual Production?
01:42 Generating a 360° image
03:17 Preparing the 3D Model [Depth Maps]
06:22 Filming the scene
08:05 Key out the actors
09:10 Generate Normal Maps
11:15 Putting it all together
14:11 Final Shots
Tools used in this video:
- Skybox Labs by Blockade Labs: skybox.blockadelabs.com
- NVIDIA Canvas: nvidia.com/de-de/studio/canvas
- ControlNet for Stable Diffusion: github.com/lllyasviel/ControlNet
- midjourney: midjourney.com
- High Resolution Depth Maps for Stable Diffusion WebUI: github.com/thygate/stable-diffusion-webui-depthmap-script
- Blender: blender.org
- CamTrackAR: apps.apple.com/us/app/camtrackar/id1502545167
- Reality OBJ to USDZ Converter: apps.apple.com/de/app/reality-obj-usdz-converter/id1603967912
- After Effects: adobe.com/products/aftereffects.html
- Runway.ml: runwayml.com
- High Reslution Normal Maps for Stable Diffusion: github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script
- DaVinciResolve: blackmagicdesign.com/products/davinciresolve
- Reference to image plane: github.com/Pullusb/reference_to_image_plane
Videos Links:
- Avatar Featurette: Performance Capture: youtube.com/watch?v=OJ1JzYPjcj0
- Avatar & Avatar 2 Behind the scenes - How James Cameron Evolved Motion Capture in the Avatar Films: youtu.be/WrKL432F86Q
- The Virtual Production of The Mandalorian Season Two: youtu.be/-gX4N5rDYeQ
- Blender Motion Tracking - Room Transformation! by Ian Hubert: youtu.be/lY8Ol2n4o4A
- Grae n YouTube Channel: youtube.com/@UC_EkMqKOm7Ej_g03Ryv0UZw
- Inpainting in Augmented Reality: youtube.com/watch?v=wJNROIpjveI
- How to Make a Blur Node in Blender!: youtube.com/watch?v=IHmN8V4Yqwk
❤️ Buy me a coffee:
ko-fi.com/mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
