Tangibility and Richness in Digital Musical Instrument Design

Expression in musical performance is deeply connected to the touch of the performer and the nuances of their control. Alongside hearing the haptic sense is the primary modality through which we engage with musical instruments, and learning a musical instrument can be understood as one of the most developed haptic cultural practices. My research interests focus on the mechanisms of touch and how they relate to the design of digital musical instruments.

When playing an acoustic instrument we get a great deal of rich sensory information from the part of our body which makes contact with the instrument (hands, fingertips, lips, shoulders). In the case of digital musical instruments, although touch-mediated interaction is still the primary means of control, there is no comparably rich physical experience from the instrument and little relationship between the sound of an instrument and its feel.

My research focuses on the contact that a performer makes with an instrument: the combination of auditory and haptic feedback that we get from an instrument. This is with the aim of informing design both in terms of the feel of an instrument, that is its immediate feedback to our haptic and auditory sense, and in terms of the mental model we have of that instrument, shaping the gestures that we use to control it and our internal representation of its mechanism. As part of my research I have designed new musical instruments to investigate different aspects of touch in digital musical instrument performance.

Publications:

  1. R. H. Jack, A. Mehrabi, t. Stockman and A. McPherson. Action-sound Latency and the Perceived Quality of Digital Musical Instruments: Comparing Professional Percussionists and Amateur Musicians. Music Perception: An Interdisciplinary Journal 36 (1), 2018.

  2. R. H. Jack, J. Harrison, F. Morreale and A. McPherson. Democratising DMIs: the relationship of expertise and control intimacy. Proc. New Interfaces for Musical Expression, Virginia, USA. 2018. (Winner of the best paper award).

  3. J. Harrison, R. H. Jack, F. Morreale and A. McPherson. When is a Guitar not a Guitar? Cultural Form, Input Modality and Expertise. Proc. New Interfaces for Musical Expression, Virginia, USA. 2018.

  4. R. H. Jack, T. Stockman and A. McPherson. Rich gesture, reduced control: the influence of constrained mappings on performance technique. 4th International Conference on Movement Computing, London, United Kingdom. 2017.

  5. R. H. Jack, T. Stockman and A.McPherson. Maintaining and Constraining Performer Touch in the Design of Digital Musical Instruments. Proc. Tangible and Embodied Interaction, Yokohama, Japan. 2017.

  6. R. H. Jack, T. Stockman and A.McPherson. Effect of latency on performer interaction and subjective quality assessment of a digital musical instrument. Proc. Audio Mostly, Norrköping, Sweden. 2016.(Nominated for best paper award).

  7. A. McPherson, R. H. Jack and G. Moro. Action-Sound Latency: Are Our Tools Fast Enough?. Proc. New Interfaces for Musical Expression, Brisbane, Australia. 2016.

  8. G. Moro, S. A. Bin, R. H. Jack, C. Heinrichs and A. McPherson. Making high-performance embedded instruments with Bela and Pure Data. Proc. ICLI, Brighton, United Kingdom. 2016.

  9. R. H. Jack, T. Stockman and A. McPherson. Navigation of pitch space on a digital musical instrument with dynamic tactile feedback. Proc. Tangible and Embodied Interaction, Eindhoven, Netherlands. 2016.

  10. R. H. Jack, A. McPherson and T. Stockman. The design of tactile musical devices for the deaf. Proc. International Conference on the Multimodal Experience of Music, Sheffield, United Kingdom. 2015.