EECS PhD student contributed to the new generative AI model in gameplay creativity 

A groundbreaking study published today in Nature presents a significant advancement in generative AI for the creative industries, particularly in video game development. The research, primarily led by Microsoft Research, explores how AI can support and enhance human creativity by aligning with the iterative and exploratory nature of game ideation. 

The paper introduces the World and Human Action Model (WHAM), a state-of-the-art generative AI model trained on human gameplay data. WHAM demonstrates three critical capabilities that support creative ideation: consistency, ensuring generated content adheres to logical game mechanics and narratives; diversity, allowing for novel gameplay sequences; and persistency, maintaining user modifications over iterations. 

The study builds on insights from a user study involving 27 professionals across the gaming industry. The research underscores how game developers require AI to support divergent thinking, where multiple creative possibilities are explored, and iterative practice, where content is refined through successive modifications. WHAM addresses these needs by generating diverse yet coherent gameplay sequences and allowing designers to guide AI outputs more effectively. 

The research has implications beyond gaming, paving the way for broader applications in creative AI, from interactive storytelling to content generation in film and music. 

One of the co-authors of the study, Marko Tot, from the School of Electronic Engineering and Computer Science, contributed to this research while completing an internship at Microsoft Research as part of his PhD at Queen Mary University of London. Reflecting on the findings, Tot remarked:  

“Working alongside Microsoft Research to develop WHAM was an incredible experience. It was a pleasure to work with the Game Intelligence team throughout the summer and to contribute to such a novel project. Having a tool that can instantly shape your ideas into an interactive game world opens up many new possibilities for game design. Testing various new ideas and making them persist in the virtual worlds with WHAM allows for rapid iteration and improves the creative flow of designers. Seeing characters come to life and seamlessly integrate into the game environment with just a button click makes the creative process more intuitive and accessible. I’m excited to see how it inspires the next generation of game creators.

It was a pleasure to work with the Game Intelligence team at Microsoft Research throughout the summer and to contribute to such a novel project. Working on cutting edge research while also connecting on a personal level, playing boardgames, karaoke, bowling, and many other things, is something that this team really excels at and makes you feel like a more than just a temporary intern. While working on WHAM was one of the outcomes, the real thing I got out of the internship were friendships that outlasted my four months there during summer. Pick out any person from this team and I've most likely spent time with them in Cambridge or London in the last week.” 

In an effort to encourage further innovation, Microsoft Research has released the WHAM model weights, an evaluation dataset, and the WHAM Demonstrator tool for public exploration and research. These resources are now available at https://huggingface.co/microsoft/wham. 

As generative AI continues to evolve, WHAM sets a new benchmark for how AI can be integrated into creative workflows, marking an important step towards human-AI collaboration in ideation and design.