DevonIn this video I take apart the demo reel, showing how it works and demonstrating behavior that occurs when the clips are modified.
During the hacked input section, the stick occasionally seems to flicker between two positions. This is due to game lag causing the overriding inputs to sometimes be delayed by a frame, resulting in some (generally alternating) frames where the original input from the demo clip shows through.
This video covers most of the interesting information, but I plan on looking into how the battle demo clips are scripted at a later date.
Demo Reel observations - Paper Mario (N64)Devon2018-12-28 | In this video I take apart the demo reel, showing how it works and demonstrating behavior that occurs when the clips are modified.
During the hacked input section, the stick occasionally seems to flicker between two positions. This is due to game lag causing the overriding inputs to sometimes be delayed by a frame, resulting in some (generally alternating) frames where the original input from the demo clip shows through.
This video covers most of the interesting information, but I plan on looking into how the battle demo clips are scripted at a later date.Gorialis のライブ配信Devon2021-12-22 | ...Arbitrary Code Execution data readout - Paper Mario (N64)Devon2021-03-11 | In case you are unaware, ACE (Arbitrary Code Execution) has been achieved within Paper Mario. This means that, by taking advantage of glitches (Refight Goomba King and Menu Storage), we can cause the game to jump execution to data we are able to manipulate (the controller inputs) to run our own instructions on the CPU.
The gist of how this works is that after refighting Goomba King, we can achieve Menu Storage, which allows Mario to move even though he is considered to be in a cutscene and has a menu open. In this state, particles are buffered, but do not actually "spawn", so by overflowing the particle table with a large amount of particles, we end up writing into memory for existing instructions. We are able to load data over these instructions that jumps the CPU execution elsewhere by precisely adjusting where Mario stands when he creates landing and walking particles, ending up with execution right where the controller inputs are stored.
On Player 1 and Player 3, nearly any instruction is possible. The 10th and 11th bit are inaccessible because they're not bound, but in MIPS this ends up being the 2nd and 3rd bit of the rs register for I-instructions and R-instructions, which isn't particularly limiting, and in J-instructions it's so high in the offset it generally will never be used. If it ever were to turn up as a problem, we could always run instructions that writes code to get around it elsewhere in memory.
We don't have as total access to the instructions covered by Player 2 and Player 4's inputs, so barring some rare occasions, there is no real reason to use them. Since we only get access to either the higher or lower short of the instruction, we usually either can only control the opcode, or only the operands, making the actual possible instruction set very slim.
For the basic setup, we just have Player 3 be a jump instruction back to Player 1's input. This locks the game in a small loop in which we can use Player 1's input to execute custom instructions. One disadvantage is that the loop executes many times per frame, but input is only polled once per frame, so any instruction used in Player 1's slot must be executable many times in sequence without side effects. For this reason, the safe approach is to use LUI, ORI, and then SW/SH/SB to store data elsewhere in memory, and then set up that stored memory to be executed once the ACE ends, as these instructions can be individually ran multiple times with the same registers with no iterative effects.
It is through the amazing work of the people in the Paper Mario Speedrunning Discord that this was possible. Here are some of their posts regarding ACE:
If you want to see the high quality encode of this TAS without all of the overlays, I have a version uploaded here: youtu.be/AJoDIKF38os
I probably won't do much further coverage of ACE unless some big changes in methodology or capability occurs. While ACE is exciting, it is pretty much a "catchall" as far as TAS runs go. All it means is that now, what was possible through editing RAM in an emulator, is technically possible on a real console, using nothing more than inputs. This doesn't let us really learn anything new about the game though, it's just a cooler way of using what we already know.Heap watch - Paper Mario (N64)Devon2020-12-27 | This video is a live readout of the usage of the dynamic memory allocation heap during gameplay.
Interestingly, the heap is not as active as one might expect. The lifetime of many objects on it appear to be tied to scene changes and map loads as opposed to having individual lifetimes. One thing that is noticeable (although not frequently occurring in this TAS) is the unloading and reloading of content during pausing and resuming the game, which is what I believe to be the key reason it takes so long on console.
Some object types that are identifiable in relation to Eldstar have been highlighted on the heap. Unfortunately, this only consists of a small subset of everything on the heap, since there are many things that interact with the heap, and it's difficult to trace to exact purpose of each object. Fortunately, it is somewhat easy to trace what some of the higher memory objects have relevance to purely by when they are allocated and destroyed.
The object types highlighted are:
Particles - Most instances of these are relatively self-explanatory. Particles are, in implementation, primarily just textures that always face the camera. Thus, some things that wouldn't necessarily be thought of as particles, such as speech bubble icons that appear above the heads of characters in cutscenes, are actually just static particles. In addition, some cutscene-specific maps tend to have effects that look like particles, but are actually just static sprites within the map itself.
Particle handles - These are handles that particles have to chunks of data relevant to how the particle appears on screen. I'm unsure as to whether this chunk contains texture data itself, or whether it just contains indices or references that the coprocessor uses to reference the texture data. The size of these is dynamic, so if it doesn't contain texture data itself, it's likely that it might contain matrices or other large data structures instead. The routines that handle them always run through mapped memory, which implies to me that the code for them might be compressed on cartridge, making it difficult to trace down the exact functions to examine their behavior.
Events - Event objects are submitted to an event queue in static memory, and events themselves are used to manage and control behavior that continues over a period of time, such as hammering, opening doors, and cutscene elements. The data within the heap holds function pointers representing callbacks relating to the events, as well as flags that seem to influence the lifetime of the event.
World objects - I've discussed these before, but in essence, they are prefabs placed within a map that have 3D models and dedicated behavior. Examples of these are save blocks, springs, signs, and chests. These always exist for the entire lifetime of a map.
Dynamic objects - Perhaps better referred to as "actors", these are objects with cylindrical hitboxes that implement common behavior such as map physics, pathfinding, and interactivity (the ability for Mario to 'speak' to them). This mostly consists of NPCs, but dynamic objects are occasionally used to fulfill other purposes, such as giving items physics, or giving world geometry the ability to 'speak'.
Unknown blocks - Any blocks of memory where I don't know what is managing it yet. This takes up most of the heap, since there are still many things that use it that I don't understand yet. Over time as things progress, I hope to identify more of these blocks, and the most likely contenders for identification are towards the higher end of memory, as it is easier to trace the cause of these allocations and distinguish them from other things going on, whereas the lower chunks of memory are usually allocated all at once as soon as the map loads, making their purpose indistinct.
If the entire heap is taken up with objects, the game will freeze. However, this is relatively unlikely to occur in practice, as while the data for these objects exist within the heap, they are often managed statically (for instance, particle pointers are kept in a static array much lower in memory). Thus, if a large amount of objects are allocated at once, it is far more likely these static arrays will overflow and induce a crash much sooner than the heap itself will fill up.
Oh, and also, the current version of the GLideN64 graphics core has some issues that the previous version didn't, you can see at places such as 33:24 that the card doesn't display correctly.Eldstar patch-back test - Paper Mario (N64)Devon2020-02-16 | This video is 3 minutes of me moving a camera around.
To clarify what's going on here, this is a special build of Eldstar that mirrors the camera from Eldstar to the game, as opposed to its usual operation, where Eldstar mirrors the game camera.
This means that we can use Eldstar's orbiting camera control and tracking features and see the results in game. I spend most of this video moving around with the mouse and the menus. (you can't see the mouse, but you can see me menuing somewhat in the top left).
I could have added this feature a lot earlier, but I always wanted Eldstar to be "TAS-safe", in that it would never modify game memory or the player's actions. That way, it can be used while recording or playing back a TAS movie without worrying about desyncs. In the future, I will probably have two versions or modes of Eldstar - a "safe" mode for TAS and a "cheats" mode for moving the camera or objects around, or maybe messing with sprites and such as well.
Please also see the wiki on how to set up BizHawk to use this: github.com/scarletcafe/eldstar/wikiLoading Zone skews - Paper Mario (N64)Devon2019-11-27 | In this video I discuss some particular behaviors regarding the skews that occur on entering a new room after performing Loading Zone Storage.
Originally, I had wanted to calculate the *exact* threshold required to get an out of bounds LZS skew, but I found that many world elements, including dynamic objects (such as NPCs), shaded or animated world objects, and even simple world geometry in large amounts, affected the timing in which Mario's own behavior cycle occurs.
Because of this, I could only calculate down enough information to get thresholds of high confidence. It is possible in an exceptionally laggy room, that the error margin may become wider, but for all rooms I have tested, they produced consistent results with the calculations shown.
Interestingly, I found that trying to fine-tune down the exact values in specific rooms showed any particular loading zone combination to have a consistent threshold within around 0.005 units of error, so in TAS, it would probably be desirable to use these calculations as a mere reference from which the real boundaries can be ascertained.Slow BHS Traversal - Paper Mario (N64)Devon2019-11-10 | In this video, I demonstrate that it is possible to scale the BHS right wall slowly using precise inputs.
This serves to demonstrate that the key factor in BHS is the floor properties exhibited by the right wall, and that it is theoretically possible to scale a wall of any size that has these properties (as the ascension can be segmented into several consecutive frame perfect jumps, which can be repeated indefinitely).
With alterations to the stick inputs, it is also possible to maintain Mario's height (with float-perfect precision, due to the nature of Mario's jump arc and the exact snapping value from the wall), or to descend the wall (by simply delaying the left input by a frame or two).State 9 storage - Paper Mario (N64)Devon2019-07-15 | I first became aware of this behavior when Glower showed it on stream, asking if I had seen it before. While I had known of state 9 from encountering it in other scenarios - primarily in relation to my movement helper scripts for BizHawk - I hadn't worked out exactly what purpose the state was for until I looked into what caused it to happen.
An interesting state, for sure, it shows just how much Nintendo cared for the movement flow that they would dedicate a state just to maintaining an animation during a fall.
I was hoping that storage of state 9 would allow us to do more interesting actions, like pausing the game or using a partner, as it would possibly be useful in the discovery of new TAS strats, but alas, it seems jumping is the only action that has an exception in this state.
One thing that might be interesting is trying to respawn Bombette with a jump in midair - Bombette has some strange behavior on respawn, so maybe it would even let us double jump.Raytrace warping - Danganronpa (PC)Devon2019-07-15 | Technically speaking, raytracing is actually the process of determining which hitbox had the first intersection with the 3D 'ray' from its origin, not the process of converting a point into such a 3D vector, but this is one of the most common uses for the procedure, and I don't think the latter has a convenient name I could fit in the title. Maybe 'Inverse Perspective Transform'?
Unfortunately, as this glitch needs a fairly accurate aim to set up, and the consequence of failure is either a text box or moving outright into the wrong room, I don't think this would be a very reliable tech for speedrunning.
It could come in useful for TAS, but I've checked and libTAS can't launch the Linux version of the game yet, and the PSP version doesn't experience this bug.
Hourglass could work for the Windows version, but it's been unmaintained for long enough that it could produce desyncs, and for a game as long as Danganronpa you *really* don't want desyncs.360° collision tracking - Paper Mario (N64)Devon2019-07-15 | I had wanted to redo the full TAS length collision map video for a while with the improvements to Eldstar, but I figured it wouldn't be that interesting to do the same video again with only minor changes - so this is what I came up with instead.
Don't worry about the walls disappearing around you, they're caused by the close clipping plane. I usually use a shorter one when rendering with Eldstar (which would prevent this), but I figured it was actually better to keep it high because it allows more opportunities to "peek" into smaller rooms, as the walls don't obscure everything so often.
If you're confused about which direction to look, look down; the controller always faces towards the game screen.
All-in-all, this took about 2 weeks to prepare all the footage, edit everything together, optimize my plugin, render and reformat. I hope you find it interesting.
I still haven't managed to break into reverse engineering the alternate camera modes; it seems like the Peach intermissions with the cutscene of Bowser's castle has its own camera mode, and there's a fairly unique swing that desyncs right at the end of the game.
The credits sequence itself ends up out of line due to the different position and sizing of the viewport. I do already know of a few memory addresses that affect the viewport, but I haven't figured out the address responsible for the difference in focal or "horizon" point of the camera.Pause Menu observations - Paper Mario (N64)Devon2019-04-07 | A few observations made about the Pause Menu, including dynamic object sprite resets and background swapping.
Background swapping is definitely one of the more interesting glitches, both due to its effects and vast difference in behavior across versions. On N64 it can make the game a corrupted, unplayable mess of colors, but on Wii U it just gives you a backdrop.
I actually stumbled across the data region for the pause menu background while trying to find the data region for the real level background. BGR-format images are pretty common among game renderers even today, but it was interesting to find that the buffer used 16-bit color instead of the usual 24-bit (or 32-bit with alpha).
The bit format for the colors is [B-G-R: 5-5-6], so most of the color fidelity of the background is allocated to the red channel, not that it makes too much difference due to the quality of the image as a whole.360° equirectangular projection testing - Paper Mario (N64)Devon2019-03-31 | A test on whether 360 degree video could be made from recorded Paper Mario TAS playback.
I created this video by playing back a short TAS I had recorded 12 different times with my camera unlock script, changing the angle of the camera each time.
After doing this, I spent some time writing an After Effects plug-in that calculates the 3D projection and view matrices of a camera from the yaw, pitch and FOV of the camera, and uses this to transform the footage into a 2:1 equirectangular projection patch.
This video is the result of the stitching and syncing of these 12 clips into a full equirectangular video.
The result ended up being pretty nice, but there is a higher likelyhood of desyncs between the individual footage patches in laggier rooms, and the skybox can be disorienting in rooms where it can move, so if I attempt this again the future, I will probably work on removing the skybox, or at the very least, its motion.Game camera visualization - Paper Mario (N64)Devon2019-03-10 | Experiments in alternative tracking techniques with a mesh standing in for the camera. While not much thought usually goes into the in-game camera movement, these tracks make it easier to see just how much the camera varies between areas and cutscenes.
Some cinematic shots still don't track perfectly even now, and strong ground shaking seems to be done by something entirely different from the standard camera parameters. There seems to be many factors in getting a perfect track, but managing to pull a view matrix direct from the game would allow for a perfect track at the expense of not being able to do interim-based tracks like shown here.[TAS] 2P CR-S01 [02] Frozen in Time - Trauma Team (Wii)Devon2019-03-04 | Moreorless a test TAS to get used to Dolphin's toolset. The actual operation starts at 0:58.
I don't often see multiplayer TAS runs, despite the interesting nature of doing so, so I gave it a try here. It was surprisingly not as difficult as I thought, but it would be nice if Dolphin allowed you to rerecord inputs for each player separately.
Also, the suturing for CR-S01 is weirdly arbitrary. It has been like that in most of the Trauma Center games, and even so I still haven't worked out how to get the sutures to behave, so it ends up being the biggest time killer in this TAS as a whole.
At 1:35 I left in part of an earlier attempt to demonstrate the weirdness: even with a uniform shape over a long distance, it didn't even stitch once.Dynamic objects - Paper Mario (N64)Devon2019-02-22 | In this video I demonstrate and discuss some previously omitted details about dynamic objects and their behavior(s).
Dynamic objects are one of the biggest causes and contributors to glitches and tricks in Paper Mario, and so understanding how they work so that they can be manipulated is useful for finding other situations where their hitbox can allow for unintentional behavior and clipping.
I was originally going to include more information on sprites and shadows, but as they are not exclusive to dynamic objects, I felt it would be better to leave that for a different video.Log Skip alternate camera angles - Paper Mario (N64)Devon2019-02-07 | This video shows a collision map and alternate camera angles for Log Skip. This should help give an idea of what actually happens during the skip, and why the specific stick angles were used.
As requested by glower.World object item clipping - Paper Mario (N64)Devon2019-01-26 | Examples showing how ceiling colliders on world objects can be used to push items through the floor. This behavior was first discovered by accident by Drogie on a glitchless speedrun.
While this trick isn't frame perfect, it is nevertheless difficult to set up because it requires the item to be thrown with a good RNG trajectory, and the hammer input is fairly precise.
There's no real reason to do this in an actual run, as succeeding makes you lose the item. It is interesting to know it's possible, though.Various camera unlocks - Paper Mario (N64)Devon2019-01-19 | A few experiments in how an unlocked camera can be manipulated to various effects.
There were a couple of things this footage run tested: - Is camera unlocking consistent? - Can camera manipulation be done dynamically? - Are there any side effects to unlocking the camera this way?
The answer to all of which ended up being "yes", in at least some extent. On the 'side effects' list, input movies sometimes desync as seen in the video, and gameplay-wise some cutscenes softlock the game if they use camera-related cues to advance.
This unfortunately makes this technique unsuitable for a fully unlocked, synced camera following TAS footage, but this kind of thing might be useful for custom ROM hacks instead.Zones & other collision types - Paper Mario (N64)Devon2019-01-10 | In this video I cover zoning of world geometry, and the other collision types I didn't elaborate on in my video on overworld surface types.
While testing and reverse engineering behavior for dynamic objects, I found that dynamic objects actually have a failsafe implemented for if Mario somehow gets directly in the center of them, giving them no obvious direction to push him out. In this case, the game defaults to pushing Mario in the Z+ direction, which is usually towards the camera.
I also found some interesting details about how the game does trignometric functions. While knowing the process is useful, it isn't incredibly interesting for the time it takes to explain, so I will only be documenting if it has a significant bearing on how a specific mechanic behaves.Game camera footage tracking - Paper Mario (N64)Devon2019-01-04 | An interesting editing test - here I track footage from Paper Mario into 3D space by using the parameters of the ingame camera. In this fashion, the point focused on by the camera (usually Mario) appears in the correct position in 3D space, although the surrounding footage is affected by parallax error.
Not that useful in practice, but an interesting visual nonetheless.Dojo Early - Paper Mario (N64)Devon2018-12-29 | In this video I discuss a known glitch called "Dojo Early", where Mario can get access to the dojo before it is properly unlocked. It works in a similar way to the Blue House Skip, using an unintentional floor to gain height over collision.
The purpose of the blockage was probably to prevent the player from taking on the dojo foes before obtaining the Action Command. After returning from Shooting Star Summit, the dojo is open, making this glitch unnecessary, although it has no bearing over a standard Any% speedrun in any case.Overworld collision types - Paper Mario (N64)Devon2018-12-23 | This video demonstrates and explains how Mario interacts with the 3 main types of overworld colliders.
Understanding how these colliders work allows better explanation of some present glitches and tricks possible in Paper Mario, as well as establishing ground for newer ones to be discovered.
As well as discussing other colliders, I will likely revisit this video at a later date to discuss more in-depth the exact processes the game goes through when handling each of the mentioned colliders, as well as give more detail on how the collider type is distinguished.Paper Mario (N64) - Wireframe map collision viewDevon2018-11-30 | This is an "updated", or rather recreated version of a previous video (youtu.be/bX3Cvy8hmyQ) showing camera tracking and wireframe views in Eldstar over game footage.
Occasionally the collision map and footage appear to lag behind eachother. This is not a bug, but rather an artifact of how the game handles lag frames, resulting in the internal game state occasionally being 1 frame out of sync with what is being shown on screen (Eldstar always represents the internal game state).
It would be nice to try and mitigate this effect in the future, but right now I am prioritizing work on further reverse engineering of other game mechanics.Paper Mario (N64) - Varied map collision testing 3Devon2018-11-18 | This is the first video demonstrating collision map footage from my realtime OpenGL implementation, featuring camera tracking and dynamic object target angles.
This system doesn't look as nice as a raytraced render, but I still plan on potentially adding shadows and SSAO as a visual aid.
There are currently some issues with how some geometry appears when it is behind other geometry due to real-time occlusion being difficult to do with translucent objects. A potential solution to this is depth-sorting the triangles, but this still might not guarantee perfect rendering if triangles overlap, and it causes the program to run a lot slower.Paper Mario (N64) - Varied map collision testing 2Devon2018-10-26 | This video expands on the previous video, covering an entire playthrough and fixing the dynamic object issue.
This pipeline is only marginally faster than the previous. I was able to mitigate some major slowdowns in rendering, but rendering out both views still took a few days. During this time, I've written a program that can hook directly into the emulator to produce a view of the collision map in C++ using OpenGL. It works pretty well, but I'd like to polish it and perhaps reverse engineer more mechanics before I showcase and release it.
Overall, I'm happy with this video. Similar to the last one, the camera angles can get a bit unwieldy, especially for rooms that are closed cubes, have strange origin positions, or are particularly wide (like hallways). In my program you can adjust the camera freely with the keyboard, but I will probably add context-sensitive automatic camera adjustment to make it better for encodes like this.Paper Mario (N64) - Varied map collision testingDevon2018-10-19 | This system is similar to the one in my previous video, but applied over a variety of maps using a segment from Malleo's all-cards TAS (you can see their encode at youtu.be/m7egGLSQ2GU).
There are some optimizations in the render script. It still takes a long time, but at least now it's manageable to the point where I can cover several minutes.
Occasionally, there are "flashes" of pink/red over the screen. These are actually huge entities spawning at large indexes - presumably to handle particles or dialogue. I will try to have these removed from any next iterations of this pipeline.Paper Mario (N64) - Overworld collision viewDevon2018-10-16 | This video summarizes the reverse engineering I've managed so far regarding collision on the Paper Mario overworld. The view itself is not rendered realtime - the collision for all frames is dumped and then imported into keyframes in Blender using scripts I've written.
The render time for the collision view with all the frame processing and triangles took around 2 hours. I could probably easily save time by reducing the shading and polygons on the hitbox cylinders as well as better optimizing how the collision is handled from the frame dumps, but for now this was just a test to see what this would look like.[TAS] Pokémon Puzzle Challenge (GBC) - 1P Marathon: Hard 99spd 99999pts in 28:30 IGTDevon2018-08-13 | This TAS improves on my last attempt by 4 minutes, 34 seconds.
This attempt didn't have as many changes as between the previous attempts. The primary difference was tweaking of inputs and state management, to speed up input and allow riskier play. Minor optimizations on the menu screen also produced a different starting RNG seed, so the layout was different.
I've uploaded the .bk2 this time if you wish to encode this TAS yourself: https://puu.sh/BdtUe/1b2f1d93c5.bk2[TAS] Pokémon Puzzle Challenge (GBC) - 1P Marathon: Hard 99spd 99999pts in 33:04 IGTDevon2018-08-12 | This TAS improves on my last attempt by 18 minutes, 44 seconds.
The most significant changes this time were:
- The B button is used more systematically, and its use ties into the flow of the algorithm. In particular, when the screen is low, the algorithm only performs stack reduction and holds the B button. Once the screen tiles have reached high enough, it switches back into solving mode, and repeat.
- The algorithm has a lot more logic than before, now prioritizing combos over other methods of solving. The most apparent result of this in this iteration is the frequency of 5/6-combos, but this change also includes dropping tiles into place as well as much more complicated logic to avoid wasting in-game time.
I wasn't expecting to return to this TAS so soon, but I got my interest back after seeing some of the RTA stategies. This revision of the algorithm still quite as optimal as it probably could be, but all of the extra code has made it run much slower than before, so optimization is probably the best step forward for now.
I'll try to upload more content other than PokéPuzzle TASes soon, once I've gotten things ready.[TAS] Pokémon Puzzle Challenge (GBC) - 1P Marathon: Hard 99spd 99999pts in 51:48 IGTDevon2018-08-05 | This TAS improves on my last attempt by 27 minutes, 44 seconds.
The most significant changes in this iteration are:
- The B button is held a lot more than the previous TAS. In the previous TAS, B was only held while nothing was queued to do. In this TAS, B is held during any inactivity regardless of any buffered movements. This is riskier than the previous behavior, but reduces the amount of idle or fidgeting time.
- Parts of the board are marked as "completed" and are actively avoided when buffering new actions. This does NOT mean when that part of the board is cleared game-wise, but rather when all sufficient pieces are there for it to be cleared even if it doesn't occur immediately. This reduces the previous stalling, so more chains are cleared in parallel, but also increases fidgeting when the script attempts to set up a new combo but can't intervene with obstructing blocks that are still being cleared.
- The script now considers the row that scrolls up from the bottom of the screen. In conjunction with the aforementioned "complete" state, this allows for combos to be buffered, which happens often within this run when it's possible.
I may revisit this script and TAS in more detail if I can find more large optimizations and time-saves, but for now I'll put this run off in favor of other projects.[TAS] Pokémon Puzzle Challenge (GBC) - 1P Marathon: Hard 99spd 99999pts in 1:19:32 IGTDevon2018-08-05 | This TAS was an experiment I did to test using BizHawk's Lua tools for TASing.
This TAS, while somewhat visually impressive, is not optimal and could be sped up by writing the algorithm to prioritize larger combos and make more generous use of the B button to speed up gameplay.
I will likely follow up this TAS with a more optimized version at a later date as well as perhaps experiment with the other gamemodes and multiplayer.