Spatial Audio Production and Reproduction

Martin Morrell and Joshua D. Reiss

Spatial audio is an area that has been developed for many years by reproduction techniques such as Ambisonics, Wave Field Synthesis and Vector Base Amplitude Panning. It is quite often difficult to fit these spatial audio rendering techniques into current audio production workflow, which leads to them not being widely adopted by the audio and music community.

A 3D audio method known as Ambisonics was modified to give the end user; musician, producer or composer, more creative freedom and extra controls. The outcome of the research produced a spatial audio system linking a standard DAW (Digital Audio Workstation) with a new spatialisation application controlled by midi commands. The videos have the audio rendered binaurally – this means the 3D audio can be heard over headphones.

How it was done (requires headphones)

The theory of variable-order, variable-decoder Ambisonics has been accepted for publication at CMMR2012 The technology is designated for 2D reproduction, originally over an octagonal speaker layout. The technology broke the limitation of using a fixed order and fixed decoder type for Ambisonics production and playback. By eliminating the B-Format transport method and encoding directly to the speakers, as say 5.1, each sound source can be panned using variable-order and variable-decoder. This system also leads to the use of sound sources that are of different orders within a single environment. The aim of the video above is to describe how the final multimedia piece `Mechanics of Love’ was developed from score to final production.

About the Full Length Demonstration (requires headphones)

Mechanics of Love was written for a piano trio and completed in 2012 by Laura Beaumont to showcase this new technology. The piece is set to an animated short film; `Invention of Love' written and directed by Andrey Shushkov. The use of the spatialisation tool allowed for a greater creativity in conveying layers within the musical scene at varying distances, widths and positions across a horizontal plane. This helped create a greater sense of foreground and background. Placing the ‘mechanical’ elements in the distance and wide when in city surroundings for example, but brought to the centre and narrowed when portraying a specific man-made character/element such as the record player, horse, dog and heart. The cello and violin parts were generally placed in the stereo field in accordance with their on-screen presence but the distance feature was employed to convey the strength of the character’s emotions, getting closer at climax points to create intensity for the listener. The tool was used most creatively in trying to achieve a sense of movement through swirling musical layers around the full range of the eight speakers for example in the rapid bustling city and spiral staircase scene, equally in the slow panning of the opening and pier scenes.

Variable-Polar Pattern Reproduction (Requires Headphones)

Following on from the development of the system above a new technology was developed called ‘Variable-Polar Pattern Reproduction’. This new technology is like similar to the one above, but the controls presented to the user are more intuitive with less dependency upon each other. This video demonstrates both technologies and their features and then shows how they can be used simultaneously together. The toolchain is presented in the video, but of special note is the use now of a VST plugin - meaning the control is less specific to a certain DAW host as well as not requiring a midi and audio track per sound source. Also the commands are now sent via OSC and not MIDI, giving a greater range of control. This new method uses the speakers to create the same type of signal as the Variable-Ambisonics but is much more computationally efficient.

Publications

M. J. Morrell, C. Baume and J. D. Reiss, "Spatial Audio System for Surround Video," to appear in International Journal of Digital Content Technology and its Applications, 2012

M. J. Morrell and J. D. Reiss, “A 2d variable-order, variable-decoder, ambisonics based music composition and production tool for an octagonal speaker layout,” 9th International Symposium on Computer Music Modeling and Retrieval (CMMR) Music and Emotions, London, June 19-22, 2012.

M. J. Morrell, J. D. Reiss and S. Wilkie, “Surround Sound Using Variable-Ambisonics and Variable-Polar Pattern Theories,” IEEE International Conference on Multimedia & Expo (ICME), Melbourne, Australia, July 9-13, 2012.

M. J. Morrell, C. Baume and J. D. Reiss, " Vambu Sound: A Mixed Technique 4-D Reproduction System with a Heightened Frontal Localisation Area", AES UK 25th Conference/4th International Ambisonics Symposium, York, March 25-27, 2012.