An exploration of multimodal method for digital game analysis

Abstract

Ever since Mitchell put into words the famous idea that “All media are, from the standpoint of sensory modality, 'mixed media'” (2005), and allegedly long before that, studies of media products have dealt with mediation as a multimodal phenomenon or function in the process of meaning-making. In today’s digitalized world, the modes of communication have shifted somewhat even though the smallest constituent parts of communication stay the same. Digital games, as they have evolved exponentially in the past half-century make use of new affordances of the digital mode to convey stories that are in dialogue with the person interacting with them. This presentation will focus on how an experimental multimodal toolbox building on and expanding the framework presented by Hawreliak (2018) can be used to analyze digital games, taking the 2022 game Stray as an example. Through the motif of decentralizing the human, it is explored how the focalizer’s perspective, a stray cat, is enhanced in the game design through the different modes of the game by looking at the visual aspects, as well as the sounds, the narrative, the haptic feedback and the gameplay mechanics and design. The field of multimodality historically focused on communication and linguistics but does today incorporate studies of a broad range of communication forms. Social semiotics and discourse analysis have shifted the focus from the media product as a separate entity and highlighted the importance of the socio-cultural context and power hierarchies in communication. Games are naturally not excepted from this, and the role of the player as interpreter in meaning-making is crucial.