Defence of dissertation in the field of Media Technology, Roberto Pugliese, M.Sc., M.Phil.

Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction

Roberto Pugliese, M.Sc., M.Phil. will defend the dissertation “Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction” on 11 December 2015 at 12 noon in Aalto University, School of Science, lecture hall TU1, Otaniementie 17, Espoo. In this dissertation “Creating and evaluating embodied interactive experiences: case studies of full-body, sonic and tactile enaction”, interactive media technology creates novel experiences for the body and the senses. A collection of case studies ranging from sonic environments, music, movement and gaming shows that engaging the body through senses and movement directly results in increased immersion, boosts performance during exercise and positively affects experience and emotions.

In a moment when technology is ubiquitous and our virtual identity scattered across social media, this thesis discusses and proposes media technology for the creation and augmentation of human experiences for the here and now. The designs implemented in this research demonstrate multimodal technology that guides the participant’s action by providing real-time feedback, being it auditory, tactile or visual. For instance, using wearable electronics sound and touch immerse the participant in an environment populated by moving sounds. Everyday objects can emit virtual sounds without the need of modifying them, creating a soundscape or speak to the person grabbing them. Mobile phones can be used as musical instruments that modify their sonic characteristics according to the behavior of the performers. The sound of jumping on a trampoline can be modified to alter the impression of its softness or give a sonic feedback of the reached jumping height. A motion-controlled avatar can affect the emotional state of a player enacting it or respond to the action of a person with recognizable behaviors that show personalities and emotional traits.

Research about e.g. sports and motor rehabilitation can directly gain from the proposed design, so does motion-based games and media art. The guiding principle is to design interaction for the human body, a complex multi-sensory active system immersed in an environment he or she engages with. The human body mediates the experience and gives meaning to it. Designing a technology for a body-in-action, i.e. enactive technology, is to take advantage of the senses and motor skills we already possess. Technology is seen as a layer between the action and perception of the human that further mediates the experience of the environment. The results show that adopting the enactive framework helps in the realization of systems that achieve natural interaction for novel experiences.

Video documentation of the case studies can be seen at: http://robertopugliese.net/Dissertation-Creating-and-evaluating-embodied-interactive-experiences

Dissertation release (pdf)

Opponent: Associate Professor Zsófia Ruttkay, Mohaly-Nagy University of Art and Design, Hungary

Custos: Professor Tapio Takala, Aalto University School of Science, Department of Computer Science

School of Science, electronic dissertations: https://aaltodoc.aalto.fi/handle/123456789/52