Tactile Feedback and instrument navigation

This first study of this research project investigated how a digital musical instrument can respond dynamically to performer control through vibrations in the body of the instrument, and how this feedback can provide the performer with additional performance information. The aim was to create vibro-tactile feedback conditions that could guide the hand of the performer whilst remaining harmonically linked to the sound output of the instrument. An embedded DMI with various vibro-tactile feedback conditions was created using Bela to test this out. It was designed to assist performer intonation (accuracy of tuning) on an instrument with continuous pitch control (like a cello, trombone, theremin). The instrument was tested with ten musicians who played the instrument while it was hidden from sight, encouraging concentration on the relationship between touch and hearing. Findings from the user study were present at TEI 2016 in Eindhoven and can be read in the paper below.

We found an increased tuning accuracy with each of the tactile feedback conditions, but noted that certain vibration patterns imposed temporal constraints that disrupted musical performance. We also witnessed a series of emergent gestures that used the interplay of audio and haptic feedback in unexpected ways. Where previous studies used force feedback to push the performer to the right pitch, this interface required an active correction by the performer. It is interesting that both methods are successful in improving accuracy which suggests that there is great potential for integrating audio-related vibrotactile feedback for guidance tasks in interfaces although the temporal demands of performance can limit the complexity of this feedback.

R. Jack, T. Stockman and A. McPherson. Navigation of pitch space on a digital musical instrument with dynamic tactile feedback. Proc. TEI, Eindhoven, Netherlands, 2016.