• SIE Communications

In support of SIE’s goal to make our products and services accessible, we continue to forge our path and pursue technical innovation. Sony Assistive Musical Instrument Hackathon in partnership with Paraorchestra, Drake Music and Watershed, held in September 2024, was the fourth in a series of similar events organized across the globe. It brought together Sony Interactive Entertainment (SIE), Sony Music Entertainment (Japan) Inc., and other Sony Group employees, university students, and members of the Paraorchestra—a professional ensemble comprised of disabled and non-disabled musicians and composers. 

Liza Bec, SIE’s team representative from Paraorchestra, has a very rare form of epilepsy that is triggered by specific movements involved in playing instruments at a professional level. Specifically, performing arpeggios and crossing hands are two performance elements that can be trouble spots for Liza. Thus, they created the Robo-recorder, a custom-built, electronically enhanced musical instrument, and the members of the SIE team were paired with Liza to expand on the device’s capabilities:

  • Alexei Smith, SIE (Team Lead)
  • Liza Bec, Paraorchestra musician
  • Chris Buchanan, SIE
  • Mike Middleton, PhD student at the University of York
  • Kim Steel, PhD student at the University of York
  • Jaylen Sezen, a student at the University of East London
  • Luke Child, PhD student at UWE Bristol

Well, it’s probably all of the above and more.

But whatever the reason for the psychological affinity that tech and science-driven folks have for video games, that personal relationship neatly parallels the correspondence in the research world between the tasks of game development and the cutting edge exploration of technologies such as artificial intelligence. For every computer science paper on the arXiv – from computer vision to LLMs to personal immersion to cognitive science and general AI – there are a half dozen use cases you can name in the creation of video games. So, it is not too much to say that the future of gaming (to say nothing of much of the future of the gaming community) can be found in today’s trailblazing university AI laboratories.

This winter, Sony Interactive Entertainment (SIE) is going to take full advantage of this synergy and further its burgeoning relationship with academia by offering an Independent Activity Period (IAP) activity at MIT. This winter term activity (MIT prefers “activity” to “course”) will consist of a brief, project-oriented survey of some of the main research topics that are integral to the development of the future of gaming. Lecturers from SIE (including yours truly) as well as contributors from MIT and other universities, will discuss topics such as player immersion, persuadable chatbots and realistic NPCs, sentiment analysis, rendering, asset and game creation with Unreal Engine and the use of LLMs for agent planning and the development of game tutor agents (all in one month! Phew!). Students will also choose from a list of hackathon-style project templates (or create a project on their own) to be demo-ed on the final day of the nine lecture series. The best hackathon project (as judged by our panel of experts) will receive a brand new PS5 Pro with a suite of some great AAA games!

Related Posts