Mickmumpitz
We Built an AI Render Engine for FREE
updated
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: patreon.com/posts/free-workflows-113743435
You can find my PATREON EXCLUSIVE advanced workflow here: patreon.com/posts/advanced-create-113745268
Lora training is easy now!
Learn how to use FLUX, ComfyUI, and SDXL to create AI characters for movies, children's books, and virtual influencers. Discover techniques for generating AI character sheets, training custom Loras, and achieving consistent results across multiple images. This guide covers advanced workflows, Flux Gym, Pinokio, and FLUX Lora training. Perfect for creators looking to elevate their AI art and storytelling capabilities.
Chapters:
00:00 Intro
00:50 Process
01:35 Flux Workflow
04:21 Advanced Versions
04:45 Limitations
05:34 SDXL Version
06:40 Preparing the Images
07:38 Flux Gym
09:31 Using the LoRA
11:46 Multiple Characters
If you like my work, please consider supporting me on Patreon:
patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: patreon.com/posts/new-video-free-112156652?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Here are two powerful AI VFX workflows for creating stunning set extensions for your video projects. The first workflow enables selective image inpainting, similar to Photoshop's generative fill but more advanced and free. The second workflow seamlessly integrates 3D models into scenes, matching lighting, colors, and style.
Learn how to use and install these free tools step-by-step to elevate your visual effects and expand creative possibilities in filmmaking and video production using the power of AI. Matte paintings and set extensions have never been so easy!
Chapters:
00:00 Intro
00:55 Preparation
01:35 ComfyUI Installation
03:37 Using the 2D Workflow
05:10 2D Workflow Walkthrough
07:11 Compositing 2D
07:35 3D Workflow Preparation
12:12 Using the 3D Workflow
15:03 Compositing 3D + Tips & Tricks
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: patreon.com/posts/new-video-free-110950228
You can find my PATREON EXCLUSIVE advanced workflow + example files here: patreon.com/posts/advanced-guide-110954469
Reddit Post: reddit.com/r/StableDiffusion/comments/1euz2a9/union_flux_controlnet_running_on_comfyui_workflow
TOOLS:
- Leonardo AI* app.leonardo.ai/?via=mickmumpitz
- blender blender.org
- Tripo* https://www.tripo3d.ai?ac=C7ZV
- ComfyUI github.com/comfyanonymous/ComfyUI
*Affiliate Links: Help support the channel at no extra cost for you if you decide to sign up!
You can use this workflow in your browser or install it locally for free and use it with your favorite Stable Diffusion model or even with the brand new, absolutely mind blowing FLUX image model by Black Forest Labs.
I’ll also show you how you can seamlessly integrate characters or other assets into these 3D environments, how to combine them with my AI compositing workflow for a full low budget virtual production environment and I also want to try traveling to these worlds using my VR headset to see if they hold up in person.
So whether you need background stages for games, digital sets for cg movies or live action vfx shots, this workflow will let you speedrun environment creation for your projects.
CHAPTERS:
00:00 Intro
00:55 Creating a scene
01:35 Render Passes
05:17 LeonardoAI
09:12 ComfyUI: SDXL
10:10 ComfyUI: FLUX
11:45 VR
12:40 Adding Characters & Assets
15:35 Virtual Production
16:14 Animated Environments
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: patreon.com/posts/new-video-free-109770732?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
You can find my PATREON EXCLUSIVE advanced workflow here: patreon.com/posts/advanced-v01-109771250?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Easily composite any video on any background with one click! I developed a free AI-powered ComfyUI workflow to automatically cut out the subject, match the lighting, fix edges, and add additional VFX. Just import video and background, wait a few minutes, and get your final shot. Fine-tune and control the result further with traditional VFX tools like Blender and After Effects, and other mind-blowing AI tools. Whether you're creating a music video or sci-fi short, this workflow can transform the way you make movies!
Tools used:
ComfyUI
BlenderAfter
Effects
Chapters:
00:00 Intro
00:51 AI Greenscreen
02:45 Relighting
04:00 Workflow (Simple)
08:36 Workflow (Advanced)
10:25 EXAMPLES
16:43 Installation
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
This Stable Diffusion workflow can be combined with my character model sheet workflow for consistency. It's great for movie pre-visualization, concept art, or graphic novel panels, since it can handle multiple characters through automatic regional prompting. To demonstrate its full potential, I'll use it with Dream Machine, Claude 3.5 Sonnet, ElevenLabs, Udio, and DaVinci Resolve to create a full AI short film about the endless fight between knights and gnomes.
You can find the free workflow & guide here: patreon.com/posts/new-video-free-108432558?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Advanced Workflow + example files: patreon.com/posts/advanced-next-ai-108435562?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Character Model Sheet Workflow: youtu.be/849xBkgpF3E?si=WTZL4L8BIx5Ep30p
Chapters
00:00 Intro
01:00 Demonstration
03:00 Installation
06:00 Simple Workflow
09:23 Advanced Workflow
10:33 Tips & Tricks
11:12 Short Film Process
13:44 Final Short Film
15:45 Outro
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
You can find the FREE WORKFLOWS & INSTALLATION GUIDE here: patreon.com/posts/new-video-make-106483196?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
You can find my PATREON EXCLUSIVE advanced guide, example files and poses here: [coming very soon!]
Check out my last video on this subject: youtu.be/849xBkgpF3E?si=ohbXy_M5dA-6KJhq
Tools used:
ComfyUI github.com/comfyanonymous/ComfyUI
midjourney midjourney.com
tooncrafter (huggingface demo) huggingface.co/spaces/Doubiiu/tooncrafter
tooncrafter in ComfyUI
Chapters:
00:00 Intro & Overview
01:00 The Tool
02:00 midjourney
02:37 Tooncrafter on Huggingface
03:04 Using Photoshop
03:37 Create a model sheet in ComfyUI
04:19 Posing the Character
06:34 Animating the character
08:02 CleanUp & Upscale
09:05 Reference to Anime
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
We'll explore the Large Language Model (LLM) ChatGPT on modeling a donut based on blendergurus famous donut tutorial. We will also test its 3D modeling capabilities on a futuristic sci-fi city and a lakeside village.
This is made possible by chatGPT's Python capabilities, which allow us to easily generate and copy complex scenes into Blender as code.
What do you think: Will we model based on prompts in the future or will we have an AI 3D co-pilot to help us create complex shader meshes, geometry node setups or layout geometry?
TOOLS USED:
- ChatGPT chatgpt.com
- blender blender.org
- udio (for the cube song) udio.com
CHAPTERS
00:00 Intro
00:37 Modeling a donut
04:02 Creating a sci-fi city
07:08 Final Renderings
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
I show you how to prepare your scene in the free 3D animation software blender (but you can also use Unreal Engine, Maya, Cinema 4D, Houdini, or most other 3D software to create your scenes…) and render it with Stable Diffusion in ComfyUI.
While last time we only used SDXL and Hotshot to render our scenes, I have now added Stable Diffusion 1.5 LCM support, which significantly reduces render time and VRAM consumption.
Free Workflows & Installation Guide: patreon.com/posts/free-workflows-104795004?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Advanced Workflows, Installation Guide & Advanced Step by Step Guide: patreon.com/posts/advanced-guide-104795708?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
CHAPTERS
00:00 Intro
00:31 Preparation & Recap
02:07 Depth Pass
03:15 Line Pass
03:59 Normal Pass
04:30 Mask Pass
05:20 ComfyUI
05:43 SDXL Image Workflow
08:12 NEW SD15 LCM Video Workflow
09:37 Models & Loras
10:16 Controlnet Tips & Tricks
11:36 Motion Loras
11:56 Lighting Pass
12:25 Re-Render Scenes with AI
13:18 ipAdapter
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
I developed this ComfyUI workflow in preparation for one of the next AI 3D rendering workflows, in which we will look at animating characters. But you can also use this workflow for many other exciting things: to create children's books, AI movies or one of these AI influencers everyone keeps talking about!
You can download the FREE workflows here: patreon.com/posts/new-video-create-103261741?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
Chapters:
00:00 Intro
01:09 Character Sheet
04:49 Loras & Midjourney
05:48 Controllable Characters
10:42 Outro
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
And follow me on Twitter: twitter.com/mickmumpitz
👉 You can download the free workflows & installation guide here: patreon.com/posts/free-workflow-ai-100595649?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
👉 Download the Advanced Workflows and Bonus Files here: patreon.com/posts/advanced-guide-100596644?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
👉 Here you can find a super long bonus tutorial for this workflow: patreon.com/posts/bonus-tutorial-100590607?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
SPECS
I tested this workflow on a machine with an RTX2070 SUPER 8GB VRAM. It works but I highly recommend better GPUs for a smoother experience!
CHAPTERS
00:00 Intro
00:36 Creating a scene in blender
02:09 Creating render passes
04:24 ComfyUI setup
06:13 Generating some images
08:44 Video workflow
10:58 Rendering a CG movie scene
SUMMARY:
For over a year I’ve been saying that AI is the future of Rendering.
So today I want to share a workflow with you that will allow you to render any 3D scene with AI in any style you want! You’ll also be able to create individual prompts for all the parts of the image allowing for full control over the final rendering whether it’s an Image or a Video!
On the way I want to create some amazing renderings for instagram and tiktok and also re-render a scene from my AI Pixar movie where I combined different generative tools to create a full short film.
The secret of this AI rendering pipeline is a combination of the free 3D animation software Blender and the node-based interface for stable diffusion called ComfyUI. The special feature of this AI workflow is that we do not transform a rendering with AI (vid2vid), but only pass on the 3D data of the scene in the form of render passes, which we use in Control Net. This means that the final rendering is completely prompt-based and we can create our own prompts for all areas of the image.
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
TOOLS USED:
- runway Gen-2 research.runwayml.com/gen2
- Pika https://pika.art/
- Animate Diff animatediff.github.io
- Stable Video Diffusion stability.ai/news/stable-video-diffusion-open-ai-video-model
- LeonardoAI leonardo.ai
- midjourney midjourney.com/home
CHAPTERS:
00:00 Intro
00:53 runway GEN-2
02:37 Stable Video Diffusion
03:37 Can you direct sora?
04:07 Animate Diff
05:31 Pika
05:54 LeonardoAI
07:21 COMPARISON
10:32 Conclusion
This video is also a good overview of some of the most popular AI video tools. So if you're unsure which one to use, I hope this can help!
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
👉 Download the FREE workflow here: patreon.com/posts/98818966?pr=true
INSTALLATION
1. Download ComfyUI: github.com/comfyanonymous/ComfyUI
2. Download ComfyUI Manager: civitai.com/models/71980/comfyui-manager
- if it is not showing up try this version: github.com/ltdrdata/ComfyUI-Manager
3. Download the MODELS: docs.google.com/document/d/10TQm76Ositv85PIxiz7DssFXH1V47Uvqi8yxui8R6AA/edit?usp=sharing
4. Import the & have fun creating amazing AI Animation!
Models shown in the Video:
Pixar: civitai.com/models/65203/disney-pixar-cartoon-type-a
Flat2d Animerge: civitai.com/models/35960/flat-2d-animerge
Find other Models: Civitai.com
CHAPTERS
00:00 Intro
00:46 ComfyUI Installation
02:25 First Example
03:02 Workflow Walk-through
06:42 No talking / No People
07:24 Changing the Style
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Download my FREE Workflow here: patreon.com/posts/free-workfow-lip-98157565
Tools used in this video:
- Synclabs: https://synclabs.so/
- Wav2Lip colab.research.google.com/github/justinjohn0306/Wav2Lip/blob/master/Wav2Lip_simplified_v5.ipynb
Chapters
00:00 Intro
00:16 Synclabs
00:51 The old method
02:00 Wav2Lip
02:56 ComfyUI Clean up
If you like my work, please consider supporting me on Patreon: patreon.com/MickmumpitzFollow me on
Twitter: twitter.com/mickmumpitz
Download my FREE Workflow here: patreon.com/posts/free-workfow-is-96696904?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
🎓 INSTALL ComyUI TUTORIALS
- Generate AI Videos With AnimateDiff & ComfyUI | Text to Animation Tutorial youtu.be/kmZ5S2X55fU?si=-el6doFdf00_RAQD
- ANIMATEDIFF COMFYUI TUTORIAL - USING CONTROLNETS AND MORE. youtube.com/watch?v=WHxIrY2wLQE
- Bonus Tutorial] Install ComfyUI + AnimateDiff + Custom Workflowspatreon.com/posts/bonus-tutorial-92141409
🖼️ MODELS I USED
- animatediff Motion Models: huggingface.co/guoyww/animatediff/tree/main
- Animate Diff Stabilized High: huggingface.co/manshoety/AD_Stabilized_Motion/tree/main
📖 CHAPTERS
00:00 Intro
00:38 The Tool & Setup
01:43 The Workflow
04:40 Customization
05:03 Results
I show you how you can combine 5 AI tools to create 3D animated AI movies!
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
TOOLS USED:
- Lumalabs GENIE lumalabs.ai/genie
- MOOTION mootion.com/?source=14 [Affiliate Link] Get 20% off their paid plans if you use my promo code: OIZIA4W7 before Feb 29 2024
- Join the MOOTION Discord: discord.com/invite/RCeg7gWk5D
- runway GEN-1 runwayml.com
- blender blender.org
- nvidia Audio2Face nvidia.com/en-us/omniverse/apps/audio2face
- Elevenlabs elevenlabs.io
- DaVinci Resolve blackmagicdesign.com/de/products/davinciresolve
CHAPTERS:
00:00 Intro
00:20 Script
01:29 Generate 3D Models
02:55 Join Skillshare today!
04:11 AI Animation
06:33 AI Facial Animation
08:09 Finalization
09:19 Conclusion
10:08 Final Movie!
Here is how I created this video: youtu.be/8xJV4Yq7nog?si=iUrwOcYxjYBM_qG3
If you like my work, please consider supporting me on Patreon: patreon.com/MickmumpitzAnd follow me on Twitter/X: twitter.com/mickmumpitz
#cursed #ai #aifilmmaking #planetearth
Mickmumpitz is a channel that is all about understanding new technologies from the worlds of AI, digital art, film and games in a playful and project-oriented way.
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
And follow me on Twitter/X: twitter.com/mickmumpitz
There are a lot of new AI video generators like Runways Gen-2, Pika Labs or moonvalley. But the most impressive one for me is AnimateDiff. It’s a framework for animating txt to image models and you can install it locally on your computer. So no weird Discord interfaces, or costly subscriptions are necessary!
👉 You can get my FREE WORKFLOWS here: patreon.com/posts/free-workflows-92142920?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link
TOOLS USED IN THIS VIDEO:
- ComfyUI github.com/comfyanonymous/ComfyUI
- AnimateDiff github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved
- Elevenlabs elevenlabs.io
- DaVinci Resolve blackmagicdesign.com/de/products/davinciresolve
TOOLS SHOWN IN THIS VIDEO:
- runway GEN - 2 research.runwayml.com/gen2
- PikaLabs https://www.pika.art/
- Moonvalley moonvalley.ai
RESOURCES + TUTORIALS:
- ComfyUI Installation: github.com/comfyanonymous/ComfyUI
- Animate Diff Installation: github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved
- Guide on which all my workflows are based: civitai.com/articles/2379/guide-comfyui-animatediff-guideworkflows-including-prompt-scheduling-an-inner-reflections-guide
- Great beginner tutorial: youtu.be/kmZ5S2X55fU?si=joYWTlDfZxH5rnVF
CHAPTERS
00:00 Intro
00:30 Concept
01:20 The Tool
02:47 ComfyUI Settings
06:04 Script
07:06 Voice + Effects
07:55 FINAL FILM!
Mickmumpitz is a channel that is all about understanding new technologies from the worlds of AI, digital art, film and games in a playful and project-oriented way.
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
And follow me on Twitter: twitter.com/mickmumpitz
The combination of Blender, a classic 3D animation tool, with AI tools like Stable Diffusion is one of my absolute favorite AI workflows because you can use it to create consistent environments for short films, games, or just start learning Blender. And it's just so much fun and super easy!
TOOLS USED IN THIS VIDEO [ALL FREE, YAY!]
blender - blender.org/download
Stable Diffusion WebUI - github.com/AUTOMATIC1111/stable-diffusion-webui
CHAPTERS
00:00 Intro
00:25 Layout
03:21 Exporting the Depth Map
05:07 Generating the image
07:22 Putting everything together
09:11 Animate the Camera + Effects
10:56 Final Results
Mickmumpitz is a channel that is all about understanding new technologies from the worlds of AI, digital art, film and games in a playful and project-oriented way.
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
And follow me on Twitter: twitter.com/mickmumpitz
Tools used in this video:
- Windows Media Player LegacyOBS Studio obsproject.com/Stable Diffusion WebUI github.com/AUTOMATIC1111/stable-diffusion-webui
- Control net github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/7784
- Splitter AI vocalremover.org/splitter-ai
- Adobe After Effects
My settings file: drive.google.com/file/d/1u9QUDRFjcmFLS0W9myA7zL2lcF5fc4hL/view?usp=sharing
Rotation Expression: drive.google.com/file/d/1CYFntzHk1pZVy47AKBi6vlt2TqS8VlEd/view?usp=drive_link
LINKS:
- Spiral Animation Tutorial: youtube.com/watch?v=5Es5MRSe_V0-
- How to Install Stable Diffusion - automatic1111: youtube.com/watch?v=kqXpAKVQDNU&list=PLXS4AwfYDUi5sbsxZmDQWxOQTml9Uqyd2-
- Stable Diffusion for Beginners: youtube.com/watch?v=nJlHJZo66UA
CHAPTERS:
00:00 Intro
01:28 The Process
02:22 Finding the right song
03:42 Setting up Deforum
06:16 Prompting
08:44 Control Net
09:26 Optical Flow
10:16 Visualization After Effects
11:49 controllism
14:30 Final Results
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Tools used in this video:
- midjourney: midjourney.com
- Stable Diffusion WebUI: github.com/AUTOMATIC1111/stable-diffusion-webui
- Adobe Creative Cloud: adobe.com
- DaVinci Resolve: blackmagicdesign.com/de/products/davinciresolve-runway GEN-2: runwayml.com
- Wav2Lip: colab.research.google.com/github/justinjohn0306/Wav2Lip/blob/master/Wav2Lip_simplified_v5.ipynb#scrollTo=Qgo-oaI3JU2u
Other resources:
- ROOP Deep Fake for A1111 - Face Swap Guide youtu.be/8SWon8erHZw
- DreamBooth for Automatic 1111 - Super Easy AI MODEL TRAINING! youtu.be/9Nu5tUl2zQw
Chapters:
00:00 Intro
00:42 Create Consistent Characters
02:11 Environments
02:58 Directing the Characters
04:00 Patreon
04:15 Generating movement
05:17 Facial Animation
06:37 Clean Up
07:58 Tips & Tricks
09:52 Result
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Tools used in this video:
- midjourney: midjourney.com
- ChatGPT: chat.openai.com
- blender: blender.org
- ICON: huggingface.co/spaces/Yuliang/ICON
- Stable Diffusion WebUI: github.com/AUTOMATIC1111/stable-diffusion-webui- Control Net
- Sad Talker: github.com/OpenTalker/SadTalker/blob/main/docs/webui_extension.md
- Deforum: github.com/deforum-art/sd-webui-deforum
- Adobe Creative Cloud: adobe.com
- DaVinci Resolve: blackmagicdesign.com/de/products/davinciresolve
-runway GEN-1: research.runwayml.com/gen1
- Wav2Lip: colab.research.google.com/github/justinjohn0306/Wav2Lip/blob/master/Wav2Lip_simplified_v5.ipynb#scrollTo=Qgo-oaI3JU2u
Other resources:
- Sad Talker (Huggingface): huggingface.co/spaces/vinthony/SadTalker
- High resolution Depth Maps for Stable Diffusion WebUI: github.com/thygate/stable-diffusion-webui-depthmap-script
- 3D model Male Figure: clara.io/view/d49ee603-8e6c-4720-bd20-9e3d7b13978a
- Blender Beginner Tutorials:Blender Beginner Tutorial - Part 1: youtube.com/watch?v=nIoXOplUvAw
The ULTIMATE GUIDE to Cameras in Blender (Beginners Start Here!) youtube.com/watch?v=TvXoSiV6YrM
- Stable Diffusion Models: civitai.com
Chapters:
00:00 Intro
00:32 Creating the characters
02:16 Generating the 3D model
04:03 Voice
04:26 Facial Animation
05:58 Body Animation
07:51 Patreon
08:25 "Re-Rendering" with GEN-1
11:44 "Re-Rendering" with DEFORUM
14:12 Results
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
My goal was to create a beautiful animated fairy tale. The script was written by ChatGPT, the images were generated using Midjourney, and the animation was created using Adobe Photoshop and After Effects.
Subscribe and follow us for more AI movie techniques!
Chapters:
00:00 Intro
00:44 Generating a fairy tale
01:57 Preparing the images in PS
07:57 Animation in After Effects
11:26 Additional Effects
17:30 Final Short Film
19:35 Outro
Tools used in this video:
Adobe Generative Fill: adobe.com/products/photoshop/generative-fill.html
ChatGPT: chat.openai.com
midjourney: midjourney.com
elevenlabs: beta.elevenlabs.io/speech-synthesis
We cover the entire workflow, from generating the 3D environment, to filming in a low-budget virtual production environment, to compositing your shot with some AI tips and tricks, like re-lighting your shot based on the environment you've created!
Chapters:
00:00 Intro
00:34 What is Virtual Production?
01:42 Generating a 360° image
03:17 Preparing the 3D Model [Depth Maps]
06:22 Filming the scene
08:05 Key out the actors
09:10 Generate Normal Maps
11:15 Putting it all together
14:11 Final Shots
Tools used in this video:
- Skybox Labs by Blockade Labs: skybox.blockadelabs.com
- NVIDIA Canvas: nvidia.com/de-de/studio/canvas
- ControlNet for Stable Diffusion: github.com/lllyasviel/ControlNet
- midjourney: midjourney.com
- High Resolution Depth Maps for Stable Diffusion WebUI: github.com/thygate/stable-diffusion-webui-depthmap-script
- Blender: blender.org
- CamTrackAR: apps.apple.com/us/app/camtrackar/id1502545167
- Reality OBJ to USDZ Converter: apps.apple.com/de/app/reality-obj-usdz-converter/id1603967912
- After Effects: adobe.com/products/aftereffects.html
- Runway.ml: runwayml.com
- High Reslution Normal Maps for Stable Diffusion: github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script
- DaVinciResolve: blackmagicdesign.com/products/davinciresolve
- Reference to image plane: github.com/Pullusb/reference_to_image_plane
Videos Links:
- Avatar Featurette: Performance Capture: youtube.com/watch?v=OJ1JzYPjcj0
- Avatar & Avatar 2 Behind the scenes - How James Cameron Evolved Motion Capture in the Avatar Films: youtu.be/WrKL432F86Q
- The Virtual Production of The Mandalorian Season Two: youtu.be/-gX4N5rDYeQ
- Blender Motion Tracking - Room Transformation! by Ian Hubert: youtu.be/lY8Ol2n4o4A
- Grae n YouTube Channel: youtube.com/@UC_EkMqKOm7Ej_g03Ryv0UZw
- Inpainting in Augmented Reality: youtube.com/watch?v=wJNROIpjveI
- How to Make a Blur Node in Blender!: youtube.com/watch?v=IHmN8V4Yqwk
❤️ Buy me a coffee:
ko-fi.com/mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Chapters:
00:00 Intro
00:33 Standard Method
01:13 Image Generation
02:42 Facial Animation
04:06 Preparing the 3D Sequence
05:56 Setting up the Shot (Blender Tutorial)
10:09 Final Shots
10:27 Outro
Tools I used in this video:
ChatGPT: openai.com/blog/chatgpt
midjourney: midjourney.com/home/?callbackUrl=%2Fapp%2F
Automatic1111 SD: github.com/AUTOMATIC1111/stable-diffusion-webui
Depth Map Extension: github.com/thygate/stable-diffusion-webui-depthmap-script
https://play.ht/
Adobe Podcast: podcast.adobe.com
Install Stable Diffusion Links:
Automatic 1111: https://github.com/AUTOMATIC1111/stab...
SD 1.5 Download: https://huggingface.co/runwayml/stabl...
Python 3.10.6: https://www.python.org/downloads/rele...
Git for Windows: gitforwindows.org
Buy me a coffee:
ko-fi.com/mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
If you want to learn how to create short films like this, check out my Tutorial: youtube.com/watch?v=PFybvdKH8hc
Buy me a coffee:
ko-fi.com/mickmumpitz
Tools I used in this video:
ChatGPT, midjourneyAutomatic 1111 Stable Diffusion WebUI + Depth Map Extension, play.ht, Adobe Podcast
Follow me on Twitter: twitter.com/mickmumpitz
If you like my work, please consider supporting me on Patreon: patreon.com/Mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
While last time I used AI to create a complete short film, including characters and animation, this time I want to focus especially on creating three-dimensional backgrounds and immersive settings using an impressive free tool called “High resolution Depth Maps for Stable Diffusion”.
Chapters:
00:00 - Intro
00:25 - Story
01:18 - Image Generation
02:27 - Generating the 3D Model (old technique)
03:15 - High resolution Depth Maps
04:47 - Setting up the scene
06:30 - Examples
08:09 - Voice Over
09:20 - FINAL FILM
10:15 - Outro
Tools I used in this video:
ChatGPT: openai.com/blog/chatgpt
midjourney: midjourney.com/home/?callbackUrl=%2Fapp%2FAutomatic1111
SD: github.com/AUTOMATIC1111/stable-diffusion-webui
Depth Map Extension: github.com/thygate/stable-diffusion-webui-depthmap-script
https://play.ht/
Adobe Podcast: podcast.adobe.com/Install
Stable Diffusion Links:
Automatic 1111: https://github.com/AUTOMATIC1111/stab...
SD 1.5 Download: https://huggingface.co/runwayml/stabl...
Python 3.10.6: https://www.python.org/downloads/rele...
Git for Windows: gitforwindows.org
- Check out this Installation Tutorial by Olivio Sarikas: youtu.be/3cvP7yJotUM
Buy me a coffee:
ko-fi.com/mickmumpitz
Follow me on Twitter: twitter.com/mickmumpitz
Tools used in this video:
Blender: blender.org
Dream Studio: beta.dreamstudio.ai/dream
AI Render: blendermarket.com/products/ai-render
EbSynth: ebsynth.com
Cited sources:
nvlabs.github.io/GANcraft
isl-org.github.io/PhotorealismEnhancement
Buy me a coffee:
ko-fi.com/mickmumpitz
INSTAGRAM: www.instagram.com/mickmumpitz/?hl=de
TIK TOK: tiktok.com/@mickmumpitz
Twitter: twitter.com/mickmumpitz
Sorry for the audio quality, this is my first project in this direction and not everything worked out as I imagined. The next ones will be better, I hope you like it anyway!
Please link me if you use this video as inspiration for your own projects!
Buy me a coffee:
ko-fi.com/mickmumpitz
INSTAGRAM: www.instagram.com/mickmumpitz/?hl=de
TIK TOK: tiktok.com/@mickmumpitz
Twitter: twitter.com/mickmumpitz
MickMumpitz is a channel that is all about understanding new technologies from the worlds of digital art, film and games in a playful and project-oriented way.
TOOLS USED IN THIS VIDEO:
Idea
plot-generator.org.uk
Screenwriting
GPT 3 - text-davinci-002 (beta.openai.com/playground)
Character Designs & Concept Art
craiyon (craiyon.com) & Midjourney (midjourney.com/home)
Character Modeling
PIFuHD (shunsukesaito.github.io/PIFuHD)
Voice Acting
Replica Studios (replicastudios.com)
Environments
Midjourney + 3dphoto.io (3dphoto.io/uploader)
Facial Animation
Nvidia Omniverse - Audio2Face (nvidia.com/en-us/omniverse/apps/audio2face)
Rigging & Animation
Adobe Mixamo (midjourney.com/home)
The collaboration was incredibly fun and I'm already looking forward to continue working with A.I. on more short films and other projects.
Soon I will release a process video explaining how exactly I used the tools to create this film. So don't forget to subscribe!
MickMumpitz is a channel that is all about understanding new technologies from the worlds of digital art, film and games in a playful and project-oriented way.
Buy me a coffee:
ko-fi.com/mickmumpitz


