Yesterday, Microsoft Xbox unveiled Muse, a cutting-edge generative AI model aimed at revolutionizing “gameplay ideation.” It’s a concept that might sound a bit elusive, but essentially, Microsoft describes it as a way to produce “game visuals, controller actions, or both.” However, don’t get too excited about this skipping over the regular game development process—its practical use is still pretty limited at this stage.
There’s some intriguing info about Muse, though. To train it, Microsoft employed H100 GPUs on a massive scale. It took around a million training updates just to stretch one second of actual gameplay into nine extra seconds of engine-accurate, simulated gameplay. The training data mainly came from existing multiplayer sessions, which is worth noting.
Instead of just running Muse on a single machine, Microsoft needed a power-heavy setup of 100 Nvidia H100 GPUs to get the job done. This approach is notably more expensive and energy-draining, producing a mere nine seconds of extra gameplay footage at an output resolution of just 300×180 pixels.
The highlight of Microsoft’s demonstration with Muse was its ability to replicate existing props and enemies in the game environment, mimicking their functions too. But considering the huge cost and energy expenditure, one can’t help but wonder why they didn’t simply utilize typical development tools to generate these elements.
It’s fascinating that Muse managed to keep track of object permanence and mirror the original game’s behavior. Yet, when you weigh it against the tried-and-true methods of game development, its final applications seem almost extravagant.
Looking forward, there’s a possibility that future versions of Muse could achieve more impressive feats. However, right now, it joins a long list of projects that aim to simulate gameplay purely through AI. While it’s commendable that engine accuracy and object permanence are still intact, the method is so inefficient for developing, testing, or even playing a game that it leaves me puzzled as to why anyone would opt for this approach, despite having thoroughly reviewed all the materials.
