〈 Previous Article ¦ List of all Papers ¦ List of all Authors ¦ Next Article 〉

Rhythm perception in the context of music and dance

Su, Yi-Huang 

Multisensory cues in a music performance – such as the sounds of the instruments and the musicians’ gestures – are important means of communication amongst ensemble musicians, which can also be used by the audience to form multimodal experiences of music. There is, however, a similar and yet less understood scenario that also involves visual observation of movements and auditory perception of music: dance. As dancers coordinate their movements with the musical rhythms, both streams of information may converge temporally into a multimodal percept from the audience’s perspective. Here, I will present some of my recent investigations in this scenario: How do observers extract temporal information from dancelike movements? Do similar processes as found in auditory rhythm perception also underlie visual perception of structured human movements? How does visually perceived rhythmicity interact with perception of auditory rhythms? In one study, we found that observers make use of the underlying periodicity in movement trajectories for temporal estimation of dancelike movements, which seems analogous to the benefit of a regular beat in processing auditory rhythms. Regarding the cross-modal effects, I will show that observing a humanlike figure bounce periodically induces a visual beat percept, which can modulate or improve beat perception of auditory rhythms in parallel. Furthermore, the profile of multisensory gain suggests the presence of an integrated audiovisual beat in rhythm perception and synchronization. Finally, the extent of temporal integration between auditory and visual rhythms appears to depend on the perceived congruency between the two streams. Together these results reveal an audiovisual interplay in the rhythm domain involving sounds and movements, which may be based on their sensorimotor correspondence in perception. It remains to be verified whether musicians, dancers, and the audience may employ (partially) overlapping cross-modal mechanisms for communication, synchronization, and perception.