All Things 3D | 2nd Teaser Trailer for "Ingenuity in VR." Now in Google 180 with AmbiX audio. (based on 8K 360) @allthings3d | Uploaded 3 years ago | Updated 2 hours ago
This trailer shows off a few features of the Unreal Engine using their stereo projection cube-map, along with an 7680 x 7680 over/under image camera to capture the terrain surrounding where Perseverance landed (in Jezero crater) in stereo "3D" 360 brought into Adobe Premiere along with two sets of six audio channels derived from a novel technique to capture 3D XYZ audio to be converted to AmbiX. Here this is dramatically shown in the illusion of high pitched whine of Ingenuity's two rotors (based on an actual recording during test flight) in front/left to rear/left and then approaching again at the end of the video from below. The Doppler effect was manually created based upon the actual shift in frequency related to Mar's atmosphere and the velocity of the observer's camera as it strafes the rover and Ingenuity since UE's built in Doppler effect was too slow and derived from Earth's normal speed of sound constant instead of Mar's much lower speed of sound. In fact we had to slow down the camera movement because its velocity initially was faster than Mar's speed of sound. Sadly UE's motion blur did not capture very well in the 360 so a little direction blur was added in Premiere. It should also be noted that the blades spin at 2400rpm or 400rps, which breaks down to 240 degrees per 1/60 of second, which close to one revolution per frame, meaning the blades don't really turn much at this capture speed, especially when you have no actual residual imager lag. :(
The goal when this is done is for the VR user to be able stage and launch Ingenuity for either a programmed flight, or to modes of user control: Through eyes of Perseverance's MastCAM-Z for standard "drone" like flight, or "FPV" mode through down looking cameras, which is what Ingenuity will use to "map" its path. Currently it will be optimized for the desktop with a "lite" version for the Oculus Quest 2 in standalone mode.
In creating the version of the craft for Unreal Engine, we broke apart the NASA - National Aeronautics and Space Administration #Perseverance & #Ingenuity to provide some articulation in the camera and rotors. We also redid the models UV and textures for only one "super" texture for PBR to improve VR performance. The goal is to provide a series of movable components to perform tasks that you program the rover to stage the initial launch of Ingenuity. (You will notice both the rotors and MastCAM structure move in this video)
The terrain is based off of NASA Jet Propulsion Laboratory (JPL) earlier images downloaded from Astropedia in 25cm orthographic image and 1m height map in their proprietary HiRISE DTM Mosaic which we scaled up to match the 25cm orthographic image creating whopping 400GB image, which we immediately cropped down to a more manageable 64K x 64K area, that will be brought into UE as for quadrant 16K x 16K UDIM texture. For this demo video, we only imported one 16K quadrant with a special terrain derived from a DEM in HiRISE TIFF, converted into a fairly high polygon mesh to capture some of the nuances of the sand drifts. The next step is to create a very high resolution terrain based on photogrammetry the surface around the rover, along with particles generation for shifting sand.
The lighting is based on a cubemap created from the an earlier 8K 360 I modified from the actual 360 from JPL.
More to come...
This trailer shows off a few features of the Unreal Engine using their stereo projection cube-map, along with an 7680 x 7680 over/under image camera to capture the terrain surrounding where Perseverance landed (in Jezero crater) in stereo "3D" 360 brought into Adobe Premiere along with two sets of six audio channels derived from a novel technique to capture 3D XYZ audio to be converted to AmbiX. Here this is dramatically shown in the illusion of high pitched whine of Ingenuity's two rotors (based on an actual recording during test flight) in front/left to rear/left and then approaching again at the end of the video from below. The Doppler effect was manually created based upon the actual shift in frequency related to Mar's atmosphere and the velocity of the observer's camera as it strafes the rover and Ingenuity since UE's built in Doppler effect was too slow and derived from Earth's normal speed of sound constant instead of Mar's much lower speed of sound. In fact we had to slow down the camera movement because its velocity initially was faster than Mar's speed of sound. Sadly UE's motion blur did not capture very well in the 360 so a little direction blur was added in Premiere. It should also be noted that the blades spin at 2400rpm or 400rps, which breaks down to 240 degrees per 1/60 of second, which close to one revolution per frame, meaning the blades don't really turn much at this capture speed, especially when you have no actual residual imager lag. :(
The goal when this is done is for the VR user to be able stage and launch Ingenuity for either a programmed flight, or to modes of user control: Through eyes of Perseverance's MastCAM-Z for standard "drone" like flight, or "FPV" mode through down looking cameras, which is what Ingenuity will use to "map" its path. Currently it will be optimized for the desktop with a "lite" version for the Oculus Quest 2 in standalone mode.
In creating the version of the craft for Unreal Engine, we broke apart the NASA - National Aeronautics and Space Administration #Perseverance & #Ingenuity to provide some articulation in the camera and rotors. We also redid the models UV and textures for only one "super" texture for PBR to improve VR performance. The goal is to provide a series of movable components to perform tasks that you program the rover to stage the initial launch of Ingenuity. (You will notice both the rotors and MastCAM structure move in this video)
The terrain is based off of NASA Jet Propulsion Laboratory (JPL) earlier images downloaded from Astropedia in 25cm orthographic image and 1m height map in their proprietary HiRISE DTM Mosaic which we scaled up to match the 25cm orthographic image creating whopping 400GB image, which we immediately cropped down to a more manageable 64K x 64K area, that will be brought into UE as for quadrant 16K x 16K UDIM texture. For this demo video, we only imported one 16K quadrant with a special terrain derived from a DEM in HiRISE TIFF, converted into a fairly high polygon mesh to capture some of the nuances of the sand drifts. The next step is to create a very high resolution terrain based on photogrammetry the surface around the rover, along with particles generation for shifting sand.
The lighting is based on a cubemap created from the an earlier 8K 360 I modified from the actual 360 from JPL.
More to come...
