Everyday Sensor System for Music Instruments:

Possibilities and usage in daily musical instrument playing, exercising and teaching

TOBIAS GROSSHAUSER, RALPH TRAUTNER

ReactiveS Sensor Research Lab, University of Applied Sciences Regensburg

GERMANY

Abstract: Methods, technics and expression in playing and teaching musical instruments means a lot of analysing of gesture, posture, pressure and movement of several parts of the human body.

For many of these parameters, the everyday use Sensor System provides a possibility to record and show most of the significant data in real time. This new possibilities of analysing and recording datasets of audio and sensors will show new ways in technology based pedagogy and even more important: Extended instruments, to create unheard and up to now unplayable music.

Practical experiments show a huge range of new ways to teach, practice and perform with violin, viola, cello, guitar, piano and double base.

The are used several new developed and extended magnetic field sensors, pressure sensors, position tracking and bending sensors. The following setup records and aligns data to the audio file and the score, making the everyday use simple for musicians, from beginners to professionals.

Key-Words: Sensor Array, Standalone Hardware, Musical Instruments, Teaching, Performance, Pressure Measurement, Data Recording, Data Alignment, Technology Enhanced Pedagogy.

1 Introduction

Sensors are widely spread in modern art, music installations and performances. But the missing link is the possibility for everyday use. Either it is too expensive, too complicated to install or both.

A lot of research is done in analysing Gesture of violin. Especially the right hand gestures are well investigated. But some parameters are under represented or yet unaccounted. The goal of the research was, to measure every parameter relating to violin, string playing and learning. Beside the demand on real time data acquisition an easy to use system, even for children was developed. Furthermore, because of the simplicity of the sensors, offline use is possible, because no complicated calculations have to be done. This means simple use in rough performance situation, even without computer, directly connected to hardware, such as synthesizers or data loggers and each parameter respectively sensor can be used and measured alone.

The developed sensor system aims at the following claims:

cheap and stable
easy to use, very low weight
easy to install and fix on instruments or human body
no influence on gestures and movements of the musicians or other users
valuable for pedagogical use and artistic expression
establish a basis for extended musical playing technics and musical expression
new simple to use and cheap instruments for beginners and children
new possibilities for advanced research on musical expression in combination with professional artists in everyday concert situations
usable with and without computer
reliable and stable, especially in live performances
every parameter can be detected autonomously

In the next section an outlook is given on the related work and the problems of existing sensors and in section 3 about the application, how it works and results of an example measurement.

2 Problem of Sensor Data in Music

Sensor data in music often are used for extending the possibilities of exiting instruments. But the measurement configurations are not yet useful for everyday usage. The above mentioned claims are not yet deployed or only used partly. The suggested hardware allows simple plug and play usage and common stereo jacks to connect different types of sensors.

Poepel shows a summarisation of the extended violins, playing with ASDSS sounds, playing with expanded existing instruments and playing with new gestures [1]. Askenfelt already measures bow motion and force with custom electronic devices [2]. A thin resistor wire is among the bow hairs to get position data and bow bridge distance with electrified strings. Paradiso uses the first wireless measurement system, two oscillators on the bow and an antenna combined with a receiver [3]. Also pressure of the forefinger and between the hair and wood. Young received pressure data from a foil strain placed in the middle of the bow [4]. Demoucron attaches accelerometers to the bow and measures the complete pressure of the bow with sensors connected to the bow hair [5].

Maestre presents a gesture tracking system based on a commercial EMF device [5]. One Sensor is clued on the bottom near the neck of the violin, a second one on the bow. Data of position, pressure by deforming the bow and relating data to this capturing can be calculated. A lot more systems exist, but mostly combined with a camera, which does not seem to be stable and reliable enough for performances.

A different approach is developed at IRCAM by Bevilaqua [6]. The sensing capabilities are added to the bow and measure the bow acceleration in realtime. A software based recognition system detects different bowing styles.

Guaus measures the bow pressure over all [7]. Sensors are fixed on the hairs of the bow on the tip and the frog. This means additional weight on the tip, which could influence professional violin playing, because of the leverage effect.

For the practical everyday use, no easy to use, modular and integrated system exists. This was the aim of the following described research.

3 Application

The system allows sensing relevant data from all stringed instruments, even partly from keyboard instruments and wind instruments. The basic set up and elaborated data in the following description are from violin. Because of the simplicity of the sensors, offline use is possible, because no complicated calculations have to be done. This means simple use in rough performance situation, even with or without computer, directly connected to computer hardware.

3.1 Sensor Set Up

The following set of sensors are used independently:

ReactiveS high frequency EMF sensor for bow- bridge distance

Plugged on the strings behind the bridge like a damper

ReactiveS high frequency EMF sensor for bow Position

Small box on the right arm, worn like a watch

pressure sensitive foils on the frog, detecting the pressure of the finger

Plugged on the bow with two snatchers

pressure sensor foils on the chin and shoulder rest

Plugged on the chin and shoulder rest with 2 snatchers, each

inclination sensor for bow rotation at the frog

Plugged on the end of the bow with hoog-and- loop fastener

Every sensor acts alone. This implies two advantages. First, only the sensor, which data are needed has to be plugged to the violin. Second, no additional calculations have to be done. They are sending the data autonomously to a small hardware, where several data formats can be calculated. The output can be connected to a computer for recording, calculation, analysing etc. or directly to MIDI or DMX Hardware like synthesizers or mixing consoles. No calibration has to be done. Usage of an existing wireless data transmission would round up the set up, but the radio transmitter would need additional batteries, which makes the system less reliable and too heavy for a unhindered playing.

3.2 The Connector Box

In the connector box, all sensor data are recorded or sent in common file formats to peripherals. This enables the user to simple plug in the output of the box to a synthesizer, mixer console or a computer. Even more, a recording function allows to record the data of the connected sensors during a performance, rehearsal or practicing. A switching matrix allows to change between different MIDI settings. This settings, such as channel and function of each sensor input, is programmed on a PC with a software tool and saved in the connector box. MIDI channels and functions are changed with the switching matrix. This allows a fast and autonomous use of different settings, often needed in performances, regardless of any computers.

3.3 Pressure Sensing

The systems for bow pressure sensing mostly are related to the complete transmitted force. The method described here enables the user to measure the force of every finger of the right hand. So the behaviour of each finger is shown in Fig. 2 and the position of each sensor in Fig. 1. Fig. 2 also shows the change of the pressure of each finger during a detache stroke. The summation of the data relies directly to the force transmitted to the bow.

Fig. 1

4 Conclusion

Many gesture parameters are explored in complex systems like mentioned above, but one not yet. A bow pressure measurement, like described below exemplary to the other used sensors.

The following diagram (Fig. 2) shows the complexity of the force during one bow stroke. The highest black line is the summation of the force of all fingers. This overall pressure varies from person to person, violin to violin. Bow to bow, etc. so the absolute data are not important, just the changes of the relation between the different pressure measurements of each finger.

Fig. 2

The force of the Middle and Ring Finger could be used to switch for example sounds of a MIDI synthesizer, because the effect to the over all bow pressure is quite negligible. As all measurements are in real time and affect more or less the sound while playing, some data can be used to control additional parameters. This are music related ones like sounds, effects or spatialisation or non music related like light show or illumination. New parameters for composition are simple available and extended pieces like in the Hyperbow project are possible [8].

The Data in general can be recorded, explored and processed with any standard music software or music programming environments, aligned to score and audio.

In pedagogical scenarios, objective data can represent complex correlations between pressure and sound, one of the most important parameters in instrument playing. Another goal, showing complicated relations simple is also reached.

The combination of traditional instruments, computer and hightech tools like new sensors could motivate a new generation of young musicians to learn with new methods they like and they are more and more used to. The industry recognised this trend, hopefully music schools do it also.

References:

[1] C. Poepel, D. Overholt. Recent Developments in Violin-related Digital Musical Instruments: Where Are We and Where Are We Going? NIME06, 6th International Conference on New Interfaces for Musical Expression, 2006.

[2] A. Askenfelt. Measurement of bow motion and bow force in violin playing, Journal of Acoustical Society of America, 80, 1986.

[3] J. A. Paradiso and N. A. Gershenfeld. Musical applications of electric field sensing, Computer Music Journal, 21:2, S 69-89, MIT Press, Cambridge, Massachusetts, 1997.

[4] D. S. Young. Wireless sensor system for measurment of violin bowing parameters, Stockholm Music Acoustics Conference, 2003.

[5] M. Demoucron, R. Caussé. Sound synthesis of bowed string instrumnets using a gesture based control of a physical model, International Conference on Noise & Vibration Engineering, 2007.

[5] E. Maestre, J. Janer, A. R. Jensenius and J. Malloch. Extending gdif for instrumental gestures: the case of violin performance, International Computer Music Conference, Submitted, 2007.

[6] F. Bevilaqua, N. Rasamimanana, E. Flety, S. Lemouton, F. Baschet. The augmented violin project: research, composition and performance report, NIME06, 6th International Conference on New Interfaces for Musical Expression, 2006.

[7] E. Guaus, J. Bonada, A. Perez, E. Maestre, M. Blaauw. Measuring the bow pressure force in a real violin performance, International Conference on Noise & Vibration Engineering, 2007.

[8] D. Young, P. Nunn, A. Vassiliev. Composing for Hyperbow:A Collaboration Between MIT and the Royal Academy of Music, NIME06, 6th International Conference on New Interfaces for Musical Expression, 2006.