Citation

Effenberg AO, Schmitz G. Ann. N. Y. Acad. Sci. 2018; ePub(ePub): ePub.

Affiliation

Institute of Sports Science, Leibniz Universität Hannover, Hannover, Germany.

Copyright

(Copyright © 2018, John Wiley and Sons)

DOI

10.1111/nyas.13693

PMID

29766509

Abstract

Many domains of human behavior are based on multisensory representations. Knowledge about the principles of multisensory integration is useful to configure real-time movement information for the online support of perceptuomotor processes (motor perception, control, and learning). A powerful method for generating real-time information is movement sonification. Remarkable evidence exists on movement-acoustic real-time information being effective in behavioral domains (music training, motor rehabilitation, sports). Here, we investigate whether and how biological motion perception can be enhanced, substituted, or modulated by kinematic sonification, with a focus on pitch coding. We work with gross motor cyclic movements and investigate the effectiveness of pitch scaling and consistent transposition on audiovisual motor perception accuracy (Experiment A). Beyond that, a new kind of audiovisual stimulus with inconsistent pitch transposition is used to produce a directed modulation of the integrated audiovisual percept (Experiment B).

RESULTS from Experiment A indicate pitch being powerful for mediating kinematic information to enhance motor perception and substituting information between perceptual modalities, even exceeding visual performance. Beyond these findings, results from Experiment B indicate that visual estimations of movement velocity can be enhanced or reduced auditorily. Movement sonification used for reshaping intermodal adjustments should be a powerful new tool for subconsciously shaping human movement patterns in the future.

© 2018 New York Academy of Sciences.

Language: en

Keywords

biological motion perception; intermodal adjustment; motor rehabilitation; movement sonification; multisensory integration; multisensory representation