Centre for Digital Music

 
 

IEEE AASP Challenge:
Detection and Classification of Acoustic Scenes and Events

Scene Classification Results

These are the results for the running of the Scene Classification task. For background information about this task set please refer to the subtask specifications and the subtask datasets. See also the WASPAA 2013 paper summarising the D-CASE challenge results and the EUSIPCO 2013 paper describing the baseline systems.

General Legend

Code Abstract Participants
CHR PDF May Chum, Ariel Habshush, Abrar Rahman, Christopher Sang
ELF PDF Benjamin Elizalde, Howard Lei, Gerald Friedland, Nils Peters
GSR PDF Jürgen T. Geiger, Björn Schuller, Gerhard Rigoll
KH PDF Johannes D. Krijnders, Gineke A. ten Holt
LTT PDF David Li, Jason Tam, Derek Toub
NHL PDF Juhan Nam, Ziwon Hyung, Kyogu Lee
NR1 PDF Waldo Nogueira, Gerard Roma, Perfecto Herrera
OE PDF Emanuele Olivetti
PE PDF Kailash Patil, Mounya Elhilali
RG PDF Alain Rakotomamonjy, Gilles Gasso
RNH PDF Gerard Roma, Waldo Nogueira and Perfecto Herrera


Results

Accuracy (%) St.Dev. Confusion Matrix
CHR_1 63.0 7.3 CSV
CHR_2 65.0 6.9 CSV
ELF 55.0 3.1 CSV
GSR 69.0 12.0 CSV
KH 55.0 4.4 CSV
LTT_1 72.0 (18.0)* 8.4 (3.9)* CSV (old CSV)*
LTT_2 70.0 (16.0)* 7.9 (4.8)* CSV (old CSV)*
LTT_3 67.0 (19.0)* 5.7 (4.8)* CSV (old CSV)*
NHL 60.0 8.2 CSV
NR1_1 60.0 10.3 CSV
NR1_2 60.0 8.7 CSV
NR1_3 59.0 12.2 CSV
OE 17.0 2.4 CSV
PE 58.0 9.0 CSV
RG 69.0 7.8 CSV
RNH_1 71.0 7.2 CSV
RNH_2 76.0 7.2 CSV
Baseline 55.0 10.0 CSV
*: Original LTT submission achieved low accuracy due to a bug in a Matlab toolbox - original results are presented in brackets.


Figure: Classification accuracy(%) for the SC task. Plot shows mean over 5-fold cross-validation with 95% confidence intervals. Dashed lines indicate (left to right): chance performance (black); baseline system performance (light blue); mean accuracy of human listener (orange). “Majority vote” is a meta-classifier using the majority decision over all submissions. Original LTT submissions shown in light gray.



Confusion matrix legend (for CSV files):

1 bus
2 busystreet
3 office
4 openairmarket
5 park
6 quietstreet
7 restaurant
8 supermarket
9 tube
10 tubestation



Back to main challenge page