Research on using natural user interfaces for a virtual musical instrument is described in this paper. This technology was initially used in video game consoles, such as the Wii. The user moves her hands as if playing an instrument, and the system understands the movements and plays the music. The system maps the hands’ configurations into music in real time. The paper shows related work and compares these systems with the one developed by the authors.

The natural user interface (NUI) is presented as a concept that learns increasingly complex interactions such as touch, voice, and motion. The authors use Leap Motion, a device that recognizes the motion of the fingers and the hands. Then, artificial neural networks are used to interpret the hand movements. The system allows the user to select the instrument (such as piano or violin), and is trained to understand the hand configuration. A couple of pages show the mathematical formalism of the movements’ representation in a computer.

An important section evaluates the system in terms of quality and efficiency. Based on the results of testing on a number of users, the system shows encouraging results. The paper details future enhancements that could improve the recognition and offer a better user experience for the virtual music players.

The research is well presented, and the explanation of the benefits is balanced with the details of the scientific work. The paper could be used by user interface designers or people interested in playing music with a computer.