Many of the classic Nintendo Entertainment System games are marvels of level design. The introductory moments of Super Mario Bros., for example, famously teach players to search for magic mushrooms by making it difficult to avoid the first one they encounter. In the decades since their initial release, those titles have been extensively explored, their every secret unveiled by avid enthusiasts.
Nevertheless, it has remained difficult to make clear, simple maps of the games and their worlds. As Joseph Osborn, a Ph.D. student in computer science at the University of California–Santa Cruz told me, the conventional method has remained largely unchanged for years.
“Traditionally, game maps are made by taking a lot of screenshots and pasting them together,” Osborn said. “Even in early video game strategy guides or things like that, you’d have somebody with a capture card or a camera set up in front of a TV, and they would have to take pictures multiple times in the level and stitch it together by hand.”
In collaboration with his colleagues Adam Summerville and Michael Mateas, Osborn set out to find an alternative to that labor-intensive and error-prone process. Their answer comes in the form of a software system called Mappy that they describe in a paper available now on the scholarly preprint service arXiv.
In essence, Mappy (not to be confused with the game of the same name) autonomously generates maps of a game’s levels (or, in some cases, its whole world), suturing together long scrolling screens and figuring out how distinct rooms connect to one another. “At a high level, what we do is look at what’s visible on screen at any given time. We record what the player could be seeing, and automatically stitch it together into a large map,” Osborn says. The process allows them to generate images that display the total makeup of a room, even as it changes in response to the player’s actions or other on-screen events.
Mappy/Capcom
Mappy is not a fully autonomous system, in that it doesn’t figure out how to actually navigate the game on its own. Osborn and his collaborators first have to feed it information from a playthrough, including data about which buttons were being pushed at any given time. But, as Osborn explains, “During Mappy’s execution, it’s automatically pressing the buttons” based on that information. It even has rules in place to make sure that what it’s seeing is actually part of the playable world and not, for example, a separate cut scene.
Though the process is made easier by the relatively simplicity of the NES’s graphics hardware, Mappy’s work is still difficult. Osborn and his co-authors write, “Mapping out a minute of play (3600 frames) takes between five and six minutes, mainly due to the expensive scrolling detection and game object tracking.” Some of those challenges would likely be amplified if they attempted to apply their method to a more complex console such as the Super Nintendo Entertainment System, which employs graphical layering to generate effects such as fog drifting by in the foreground. Nevertheless, they argue that similar techniques should still be feasible on other game systems.
Mappy/Nintendo
Osborn imagines an array of possible applications for Mappy. For one, he suggests, it might help those who are experimenting with procedurally generated game levels, giving them a consistent dataset from which to train machine learning algorithms that could produce playable worlds of their own. He also proposes that it might empower a kind of search engine that lets you look up a specific section of a game and then leap directly to that moment, letting you play through your memories.
Whether or not Mappy gets us there, it already stands as a charming and impressive accomplishment. As it demonstrates, modern computer science still has a great deal to teach us about the cartography of our digital past.
