A Study on Potential of Integrating Multimodal Interaction into Musical Conducting Education

USICAL conducting is an art. It is a traditional technique by using variety of hand gestures to change the music playback such as music tempo, music beat, volume, music balance between musical in-struments…etc. Music conductors, the person who are profession in musical conducting, could conduct an orchestra based on their imagination inside their mind. As people said, "the same song played by five conductors will give you five different tastes…" This is because each conductor's interpretation and expression of music is different. Music conductors based on their understanding of the music and conducting experience, using their own way to express the music to audiences. [1][2] [3] However, no matter how good are the music conductors, the conducting language is the same. This conducting had been under a few hundred years of development. It has become a very effective musical conducting language and it is also the only musical conducting language. In other words, any conductors can conduct any orchestra from anywhere since the conducting technique and conducting language are same. All orchestra can understand the meaning of musical conductors since they are using the same conducting language. [2] [3] This paper motivate by discussing the potential designing computerize basic musical conducting education with multimodal interface. In this paper, some related research and technology is analyzed. At the same time, in order to design a set of musical conducting education, 6 music conductors had been interview to share their experience on learning/teaching basic musical conducting techniques. Lastly, the paper also tries to analyze the re-sults of interview.

The invention of computer is to improve human’s work efficiency. Input devices from the initial punch-card method have been developed to current keyboard and mouse. However, these devices are still imperfect. This is because human being does not using such devices to communicate with each other. Therefore mankind must spend some time to learn how to control a computer. [4] [6] Computer becomes more and more intelligent. It seems that the traditional keyboard and mouse cannot effectively control the computer. Thus, the idea of multimodal becomes more important. It is a technique by using difference of modalities to communicate and command computers. This is because the most primitive means of communication among human is through conversation. Human are using speech and body language to express and sharing their ideas/information to each others. Today’s technology is well developed and it is no longer impossible that human can directly communicate with computer via speech and body gestures. [5][6] [7] Multimodal interaction enables to digitalize some real world activities. It brings the actual world activity into digital world. This is because multimodal enable user to use their six senses as input devices to command computer directly instead of keyboard and mouse. It is possible to use the idea of hand gesture interaction and speech interaction as the main input to conduct computers. So, if a user is using musical conducting language to command computers, computers should also able to respond users in music. This is just like what conductors do to conduct an orchestra in the real world.

And, by analyzing and monitoring the user’s conduct gestures, computer can compares these gestures with the standard musical conducting language and give comment between the differences. These comments will help to

Currently the musical conducting education is not exist yet but there are some researches on digitalize musical conducting. The main ideas of these researches are to change the music playback from computer by musical conducting.

Among those computer music researches, Mathew’s Radio Baton was the first musical conducting system that appeared in 1991. It was the first documented system that enabled the possibility of interactive conducting. It uses the movement of one or more batons emitting radio frequency signal above a metal plate to determine conducting gestures. A beat is counted when the baton goes below a certain vertical position. A MIDI file is played back synchronously with these musical beat. [8] In year 1996, Teresa Marrin Nakra collaborated with Joseph et al from MIT Media Laboratory and had designed Digital Baton to conduct Tod Machover’s Brain Opera, a MIDI-based music system. [9] Digital baton measures additional parameters such as pressure on parts of the handle to allow richer expression. The baton was a handheld device that incorporates several sensing modes. It was designed to be used like a traditional conducting baton by practiced performers. Teresa then furthered her research on Conductor Jacket project [10]. She installed 16 sensors in the conductor’s costume. Besides of the general data like acceleration of waving hands, Conductor’s Jacket also detect the size of the conductors’ gesture and their muscle tightness. These data greatly improve th

…(Full text truncated)…

This content is AI-processed based on ArXiv data.