Automatic mixing tools for audio and music production

News! Listen to audio examples from the automatic panner evaluation.

News! An article on our automatix mixing work appeared in New Scientist, Jan. 30 2010. An alternate version of the article is also available online.

On April 2nd, 2008, New Scientist broke the story of our work on acoustic feedback prevention, and original articles and interviews quickly appeared on BBC Radio 4, BBC World Service, in the Guardian's main section, Radio Deutsche Welle, ITN News and LBC's Morning Report.

Enrique Perez Gonzalez and Joshua D. Reiss

The Automatic Mixing Tools project aims to implement several independent systems that when combined together can generate an automatic sound mix out of an unknown set of multi-channel inputs. The research explores the possibility of reproducing the mixing decisions of a skilled audio engineer with minimal or no human interaction. The research is restricted to non-time varying mixes for large room acoustics. This research has application to live sound music concerts, remote mixing, recording and postproduction as well as live mixing for interactive scenes.

Demonstration videos

	Automatic panner- A video of an 8 channel mix automatically panned with no human intervention. Listen to audio examples from the automatic panner evaluation.
	Spectral enhancer- A system which ducks sources in a spectral dependent manner is demonstrated. A masking improvement measurement has been integrated into the algorithm
	Feedback prevention- Videos show how the technique helps prevent acoustic feedback during live mixing. Notice how the system immediately goes to feedback once the algorithm is turned off.
	Time offset correction- A method of automatically aligning multiple inputs to prevent destructive interference
	Automatic gain and fader adjustment for live mixing.
	Two examples of automatic equalisation. Song 1: no EQ and with autoEQ Song 2: no EQ, with autoEQ, and with all automatic tools on
	Combined automatic mixer- A demonstration of automatic mixing combining automatic gain, fader adjustment, panning and feedback prevention

The automatic mixer research distinguishes the engineering mixing from the subjective mixing. Therefore the current research is focused on a constrained rule mixing layer and a subjective mixing layer. The rule-based section is based on engineering constraints while the subjective layer is based on a target mixing style. This target style can be extracted from previously mixed songs based on feature extraction. Two approaches are under study. One is a modified automatic mixer, whose settings can be adapted based on target features and the other is based on a multilayer feedback network. The target mixing methods rely on output feature similarity to the reference features of the target mix. It is the current belief of the author that the use of expert training data can be used to increase the convergence rate of the system.

Currently automated mixers are capable of saving a timeline of static mix scenes, which can be loaded for later use. But they lack the ability to adapt to a different room or to a different set of inputs. In other words, they lack the ability to automatically taking mixing decisions. In the current research approach the starting point is a target mixing style, rather than a fixed prerecorded setup. This has the advantage of being able to blend a mixing style of a completely different song into an unknown set of inputs.

The justification of this research is the need of non-expert audio operators and musicians to be able to achieve a quality mix with minimal effort. Currently mixing is a task which requires grate skills, practice and can be sometime tedious. For the professional mixing engineer this kind of tool will reduce sound check time and will prove useful in multiple music group and festivals where changing from one group to another should be done really quickly. Currently large audio productions tend to have hundreds of channels, being able to group some of those channels into an automatic mode will ease the mixing task to the audio engineer. There is also the possibility of applying this technology to remote mixing applications where latency is too large to be able to interact with all aspects of the mix

This research is pursuing the knowledge required to develop automatic mixtures comparable in quality to those performed by professional human mixing console operators. Implementation, subjective comparison and error distance measure between a target mixture style and the automatic mixture will measure the success of the results. By style we refer not only to a certain genre of music but also to a producer or engineer subjective contribution to a mix.

Publications

Enrique Perez Gonzalez and J. D. Reiss, "Automatic equalization of multi-channel audio using cross-adaptive methods", Proceedings of the 127th AES Convention, New York, October 2009

Enrique Perez Gonzalez, Josh Reiss "Automatic Gain and Fader Control For Live Mixing", IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, New York, October 18-21, 2009

Enrique Perez Gonzalez, Josh Reiss "Determination and correction of individual channel time offsets for signals involved in an audio mixture", 125th AES Convention, San Francisco, USA, October 2008

Enrique Perez Gonzalez, Josh Reiss, "Improved control for selective minimization of masking using interchannel dependency effects", 11th International Conference on Digital Audio Effects (DAFx), September 2008

E. Perez_Gonzalez and J. Reiss, "An automatic gain normalisation technique with applications to audio mixing", Proceedings of the Audio Engineering Society 124th Convention Amsterdam, The Netherlands, 2008.

E. Perez_Gonzalez and J. Reiss, "Automatic mixing: live downmixing stereo panner", In Proceedings of DAFx-07, Bordeaux, France, 2007.

Enrique Perez Gonzalez and Joshua D. Reiss, "Anti-feedback device," UK patent GB0808646.4, filed June 13, 2008