〈 Previous Article ¦ List of all Papers ¦ List of all Authors ¦ Next Article 〉

Enabling synchronization: Auditory and visual modes of communication during ensemble performance

Bishop, Laura and Goebl, Werner 

Ensemble musicians exchange nonverbal auditory and visual cues (e.g. breathing, head nods, changes in tempo/dynamics) during performance to make their intentions more predictable and help enable synchronization. The predictability of performers' intentions is higher in some musical contexts (e.g. within phrases) than in others (e.g. following held notes or long pauses), and recent research suggests that musicians' use of auditory and visual cues may change throughout a performance as the predictability of co-performers' intentions fluctuates. We present two studies that investigate the nature of cues given in high-predictability and low-predictability musical contexts, and that test musicians' abilities to use these cues during duet performance. Study 1 tested pianists' reliance on auditory and visual cues in musical contexts where timing was more or less precisely specified by the score. Pianists performed the secondo part to three duets with recordings of pianists or violinists playing the primo parts, as the presence and absence of auditory and visual visual signals from the primo were manipulated. Asynchronies between primo recording and participant secondo performances were then calculated. The results showed increased reliance on visual cues when uncertainty about co-performers' intentions was high (i.e. at re-entry points following long pauses), but a strong reliance on auditory cues otherwise. Study 2 used motion capture to map the head and hand gestures that pianists and violinists use to cue each other in at the starts of pieces. This study is currently ongoing, but forwards-backwards head acceleration is hypothesized to indicate the timing of starting note onsets, and gesture duration is hypothesized to indicate piece tempo. This research aims to enhance our understanding of which cues and modes of communication are important across different performance contexts, and is expected to benefit ongoing efforts to develop an intelligent accompaniment system capable of responding to human performance cues in real-time.