Automatic mixing tools for audio and music production
This project aims to implement several independent systems that when combined together can generate an automatic sound mix out of an unknown set of multi-channel inputs. The research explores the possibility of reproducing the mixing decisions of a skilled audio engineer with minimal or no human interaction.This research has application to live sound music concerts, remote mixing, recording and postproduction as well as live mixing for interactive scenes.
See youtube videos at www.youtube.com/user/IntelligentSoundEng
The automatic mixer research distinguishes the engineering mixing from the subjective mixing. Therefore the current research is focused on a constrained rule mixing layer and a subjective mixing layer. The rule-based section is based on engineering constraints while the subjective layer is based on a target mixing style. This target style can be extracted from previously mixed songs based on feature extraction. Two approaches are under study. One is a modified automatic mixer, whose settings can be adapted based on target features and the other is based on a multilayer feedback network. The target mixing methods rely on output feature similarity to the reference features of the target mix. It is the current belief of the author that the use of expert training data can be used to increase the convergence rate of the system.
Currently automated mixers are capable of saving a timeline of static mix scenes, which can be loaded for later use. But they lack the ability to adapt to a different room or to a different set of inputs. In other words, they lack the ability to automatically taking mixing decisions. In the current research approach the starting point is a target mixing style, rather than a fixed prerecorded setup. This has the advantage of being able to blend a mixing style of a completely different song into an unknown set of inputs.
The justification of this research is the need of non-expert audio operators and musicians to be able to achieve a quality mix with minimal effort. Currently mixing is a task which requires grate skills, practice and can be sometime tedious. For the professional mixing engineer this kind of tool will reduce sound check time and will prove useful in multiple music group and festivals where changing from one group to another should be done really quickly. Currently large audio productions tend to have hundreds of channels, being able to group some of those channels into an automatic mode will ease the mixing task to the audio engineer. There is also the possibility of applying this technology to remote mixing applications where latency is too large to be able to interact with all aspects of the mix
This research is pursuing the knowledge required to develop automatic
mixtures comparable in quality to those performed by professional human
mixing console operators. Implementation, subjective comparison and
error distance measure between a target mixture style and the automatic
mixture will measure the success of the results. By style we refer not
only to a certain genre of music but also to a producer or engineer
subjective contribution to a mix.
S. Mansbridge, S. Finn, J. D. Reiss, "Implementation and Evaluation of Autonomous Multi-Track Fader Control," 132nd Audio Engineering Society Convention, Budapest, April 26-29, 2012
E. Perez Gonzalez and J. D. Reiss, "Automatic equalization of multi-channel audio using cross-adaptive methods", Proceedings of the 127th AES Convention, New York, October 2009
E. Perez Gonzalez, Josh Reiss "Automatic Gain and Fader Control For Live Mixing", IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, New York, October 18-21, 2009
E. Perez Gonzalez, Josh Reiss "Determination and correction of individual channel time offsets for signals involved in an audio mixture", 125th AES Convention, San Francisco, USA, October 2008
E. Perez Gonzalez, Josh Reiss, "Improved control for selective minimization of masking using interchannel dependency effects", 11th International Conference on Digital Audio Effects (DAFx), September 2008
E. Perez_Gonzalez and J. Reiss, "An automatic gain normalisation technique with applications to audio mixing", Proceedings of the Audio Engineering Society 124th Convention Amsterdam, The Netherlands, 2008.
E. Perez_Gonzalez and J. Reiss, "Automatic mixing: live downmixing stereo panner", In Proceedings of DAFx-07, Bordeaux, France, 2007.
E. Perez Gonzalez and Joshua D. Reiss, "Anti-feedback device,"
UK patent GB0808646.4, filed June 13, 2008