An immersive virtual musical instrument, or immersive virtual environment for music and sound, represents sound processes and their parameters as 3D entities of a

virtual reality Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), educ ...

so that they can be perceived not only through auditory feedback but also visually in 3D and possibly through tactile as well as haptic feedback, using 3D

interface metaphor In user interface design, an interface metaphor is a set of user interface visuals, actions and procedures that exploit specific knowledge that users already have of other domains. The purpose of the interface metaphor is to give the user instanta ...

s consisting of

interaction technique An interaction technique, user interface technique or input technique is a combination of hardware and software elements that provides a way for computer users to accomplish a single task. For example, one can go back to the previously visited pa ...

s such as navigation, selection and manipulation (NSM). It builds on the trend in

electronic musical instruments Electronic may refer to: *Electronics, the science of how to control electric energy in semiconductor * ''Electronics'' (magazine), a defunct American trade journal *Electronic storage, the storage of data using an electronic device *Electronic co ...

to develop new ways to control sound and perform music such as explored in conferences like NIME.

State of the art

Florent Berthaut created a variety of 3D reactive widgets involving novel representations of musical events and sound, that required a special 3D input device to interact with them using adapted 3D interaction techniques. Jared Bott created an environment that used 3D spatial control techniques as used in known musical instruments, with symbolic 2D visual representation of musical events. Richard Polfreman made a 3D virtual environment for musical composition with visual representations of musical and sound data similar to 2D composition environments but placed in a 3D space. Leonel Valbom created a 3D immersive virtual environment with visual 3D representations of musical events and

audio spatialization Spatial music is composed music that intentionally exploits sound localization. Though present in Western music from biblical times in the form of the antiphon, as a component specific to new musical techniques the concept of spatial music (''Raumm ...

with which could be interacted using NSM interaction techniques. Teemu Mäki-Patola explored interaction metaphors based on existing musical instruments as seen in his Virtual Xylophone, Virtual Membrane, and Virtual Air Guitar implementations. Sutoolz from su-Studio Barcelona used real time 3D video games technology to allow a live performer to construct and play a fully audio visual immersive environment. Axel Mulder explored the sculpting interaction metaphor by creating a 3D virtual environment that allowed interaction with abstract deformable shapes, such as a sheet and a sphere, which parameters were mapped to sound effects in innovative ways. The work focused on proving the technical feasibility of 3D virtual musical instruments. Gestural control was based on 3D object manipulation such as a subset of prehension. Early work was done by

Jaron Lanier Jaron Zepel Lanier (, born May 3, 1960) is an American computer scientist, visual artist, computer philosophy writer, technologist, futurist, and composer of contemporary classical music. Considered a founder of the field of virtual reality, La ...

with his Chromatophoria band and separately by Niko Bolas who developed the Soundsculpt Toolkit, a software interface that allows the world of music to communicate with the graphical elements of virtual reality.

References

External links

Immersive virtual musical instruments
{Dead link, date=January 2023 , bot=InternetArchiveBot , fix-attempted=yes
Virtual Air Guitar

Virtual Musical Instruments

sutoolz 1.0 alpha: 3D software music interface

SU-TOOLZ Interactive 3D Sound-Scape software
Human–computer interaction Electronic musical instruments Virtual reality