EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation

Julius Richter, Yi-Chiao Wu, Steven Krenn, Simon Welker, Bunlong Lay, Shinji Watanabe, Alexander Richard, Timo Gerkmann

Abstract

We release the EARS (Expressive Anechoic Recordings of Speech) dataset, a high-quality speech dataset comprising 107 speakers from diverse backgrounds, totalling in more than 100 hours of clean, anechoic speech data. The dataset covers a large range of different speaking styles, including emotional speech, different reading styles, non-verbal sounds, and conversational freeform speech. We benchmark various methods for speech enhancement and dereverberation on the dataset and evaluate their performance through a set of instrumental metrics. In addition, we conduct a listening test with 20 participants for the speech enhancement task, where a generative method is preferred. We introduce a blind test set that allows for automatic online evaluation of uploaded data. Dataset download links and automatic evaluation server can be found online.

EARS Dataset

The EARS dataset is characterized by its scale, diversity, and high recording quality. In Table 1, we list characteristics of the EARS dataset in comparison to other speech datasets.

Table 1: Speech datasets. In contrast to existing datasets, the EARS dataset is of higher recording quality, large, and more diverse. ^†contains files with limited bandwidth.

	hours	speakers	sample rate
DNS (LibriVox) [1]	556	1948	48 kHz†
LibriSpeech [2]	982	2484	16 kHz
LJSpeech [3]	24	1	22.05 kHz
TIMIT [4]	5	632	16 kHz
VCTK [5]	44	110	48 kHz
WSJ0 [6]	29	119	16 kHz
EARS (ours)	100	107	48 kHz

EARS contains 100 h of anechoic speech recordings at 48 kHz from over 100 English speakers with high demographic diversity. The dataset spans the full range of human speech, including reading tasks in seven different reading styles, emotional reading and freeform speech in 22 different emotions, conversational speech, and non-verbal sounds like laughter or coughing. Reading tasks feature seven styles (regular, loud, whisper, fast, slow, high pitch, and low pitch). Additionally, the dataset features unconstrained freeform speech and speech in 22 different emotional styles. We provide transcriptions of the reading portion and meta-data of the speakers (gender, age, race, first language).

Audio Examples

Here we present a few audio examples from the EARS dataset.

p002/emo_adoration_sentences.wav

p008/emo_contentment_sentences.wav

p010/emo_cuteness_sentences.wav

p011/emo_anger_sentences.wav

p012/rainbow_05_whisper.wav

p014/rainbow_04_loud.wav

p016/rainbow_03_regular.wav

p017/rainbow_08_fast.wav

p018/vegetative_eating.wav

p019/vegetative_yawning.wav

p020/freeform_speech_01.wav

Benchmarks

The EARS dataset enables various speech processing tasks to be evaluated in a controlled and comparable way. Here, we present benchmarks for speech enhancement and dereverberation tasks.

EARS-WHAM

For the task of speech enhancement, we construct the EARS-WHAM dataset, which mixes speech from the EARS dataset with real noise recordings from the WHAM! dataset [7]. More details can be found in the paper.

Results

Table 2: Results on EARS-WHAM. Values indicate the mean of the metrics over the test set. The best results are highlighted in bold.

	POLQA	SI-SDR	PESQ	ESTOI	DNSMOS
Noisy	1.71	5.98	1.24	0.49	2.74
Conv-TasNet [8]	2.73	16.93	2.31	0.70	3.47
CDiffuSE [9]	1.81	8.35	1.60	0.53	2.87
Demucs [10]	2.97	16.92	2.37	0.71	3.66
SGMSE+ [11]	3.40	16.78	2.50	0.73	3.88

Audio Examples

Here we present audio examples for the speech enhancement task. Below we show the noisy input, processed files for Conv-TasNet [8], CDiffuSE [9], Demucs [10], SGMSE+ [11], and the clean ground truth.

Select an audio file:    p102/00252_4.9dB p103/00464_11.8dB p104/00814_11.3dB p105/00735_0.4dB p106/00049_5.4dB p107/00519_8.1dB p102/00404_4.0dB p103/00642_9.7dB p104/00014_6.9dB p105/00246_3.8dB p106/00060_0.7dB p107/00159_-0.8dB p102/00189_2.4dB p103/00483_1.5dB p104/00357_0.5dB p105/00574_11.4dB p106/00291_5.8dB p107/00142_6.0dB p102/00327_1.9dB p103/00051_11.0dB p104/00507_0.6dB p105/00655_14.3dB p106/00852_14.9dB p107/00575_15.6dB p102/00785_6.2dB p103/00595_5.8dB p104/00662_3.4dB p105/00148_11.6dB p106/00039_3.5dB p107/00173_5.9dB

Noisy:

Conv-TasNet [8]:

CDiffuSE [9]:

Demucs [10]:

SGMSE+ [11]:

Clean:

Blind test set

We create a blind test set for which we only publish the noisy audio files but not the clean ground truth. It contains 743 files (2 h) from six speakers (3 male, 3 female) that are not part of the EARS dataset and noise especially recorded for this test set.

Results

Table 3: Results for the blind test. Values indicate the mean of the metrics over the test set. The best results are highlighted in bold.

	POLQA	SI-SDR	PESQ	ESTOI	DNSMOS
Noisy	1.81	6.48	1.28	0.57	2.79
Conv-TasNet [8]	2.68	16.56	2.41	0.75	3.43
CDiffuSE [9]	1.93	8.22	1.64	0.59	2.92
Demucs [10]	3.03	16.81	2.50	0.76	3.62
SGMSE+ [11]	3.35	16.43	2.59	0.78	3.79

Audio Examples

Here we present audio examples for the blind test set. Below we show the noisy input, processed files for Conv-TasNet [8], CDiffuSE [9], Demucs [10], and SGMSE+ [11].

Select an audio file:    00620 00377 00406 00267 00294 00252 00065 00399 00000 00474 00488 00412 00226 00520 00652 00470 00561 00371 00388 00178 00508 00711 00202 00351 00437 00303 00143 00262 00282 00613

Noisy:

Conv-TasNet [8]:

CDiffuSE [9]:

Demucs [10]:

SGMSE+ [11]:

Evaluation on real-world data

This demo showcases the denoising capabilities of SGMSE+ [11] trained using the EARS-WHAM dataset. The red frame represents the noisy input audio, while the green frame indicates the enhanced, noise-reduced output.

Dereverberation (EARS-Reverb)

For the task of dereverberation, we use real recorded room impulse responses (RIRs) from multiple public datasets [12, 13, 14, 15, 16, 17, 18]. We generate reverberant speech by convolving the clean speech with the RIR. More details can be found in the paper.

Results

Table 4: Results on EARS-Reverb. Values indicate the mean of the metrics over the test set. The best results are highlighted in bold.

	POLQA	SI-SDR	PESQ	ESTOI	MOS Reverb
Reverberant	1.75	-16.17	1.48	0.52	2.99
SGMSE+ [11]	3.61	5.79	3.03	0.85	4.73

Audio Examples

Here we present audio examples for the dereverberation task. Below we show the reverberant input, processed files for SGMSE+ [11], and the clean ground truth.

Select an audio file:    p102/00328_0.64 p103/00296_0.61 p104/00226_0.76 p105/00067_0.43 p106/00374_0.22 p107/00111_0.30 p102/00086_0.25 p103/00259_0.22 p104/00423_0.90 p105/00574_0.64 p106/00253_0.63 p107/00097_0.26 p102/00007_1.79 p103/00667_0.92 p104/00509_0.60 p105/00336_0.51 p106/00251_0.30 p107/00140_0.41 p102/00447_0.20 p103/00516_0.70 p104/00529_0.23 p105/00368_0.82 p106/00352_1.04 p107/00682_0.56 p102/00537_0.30 p103/00479_0.68 p104/00292_1.66 p105/00733_0.46 p106/00028_0.30 p107/00092_0.19

Reverberant:

SGMSE+ [11]:

Clean:

Citation

If you use the dataset or any derivative of it, please cite our paper:

@inproceedings{richter2024ears,
    title={{EARS}: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation},
    author={Julius Richter and Yi-Chiao Wu and Steven Krenn and Simon Welker and Bunlong Lay and Shinjii Watanabe and Alexander Richard and Timo Gerkmann},
    booktitle={ISCA Interspeech},
    pages={4873--4877},
    year={2024}
}

References

1. [1]H. Dubey, A. Aazami, V. Gopal, B. Naderi, S. Braun, R. Cutler, H. Gamper, M. Golestaneh, and R. Aichner, “ICASSP 2023 deep noise suppression challenge,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2023.
2. [2]V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2015, pp. 5206–5210.
3. [3]K. Ito and L. Johnson, “The LJ Speech Dataset,” 2017. [Online]. Available: https://keithito.com/LJ-Speech-Dataset/
4. [4]J. S. Garofolo, “TIMIT acoustic phonetic continuous speech corpus,” Linguistic Data Consortium, 1993.
5. [5]J. Yamagishi, C. Veaux, and K. MacDonald, “CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),” 2019. [Online]. Available: https://datashare.ed.ac.uk/handle/10283/3443
6. [6]J. S. Garofolo, D. Graff, D. Paul, and D. Pallett, “CSR-I (WSJ0) Complete - Linguistic Data Consortium,” 1993. [Online]. Available: https://catalog.ldc.upenn.edu/LDC93s6a
7. [7]G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “WHAM!: Extending speech separation to noisy environments,” in ISCA Interspeech, 2019, pp. 1368–1372.
8. [8]Y. Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 8, pp. 1256–1266, 2019.
9. [9]Y.-J. Lu, Z.-Q. Wang, S. Watanabe, A. Richard, C. Yu, and Y. Tsao, “Conditional diffusion probabilistic model for speech enhancement,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2022, pp. 7402–7406.
10. [10]S. Rouard, F. Massa, and A. Défossez, “Hybrid transformers for music source separation,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2023.
11. [11]J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based generative models,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2351–2364, 2023.
12. [12]J. Eaton, N. D. Gaubitch, A. H. Moore, and P. A. Naylor, “Estimation of room acoustic parameters: The ACE challenge,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 10, pp. 1681–1693, 2016.
13. [13]M. Jeub, M. Schafer, and P. Vary, “A binaural room impulse response database for the evaluation of dereverberation algorithms,” in IEEE International Conference on Digital Signal Processing, 2009.
14. [14]K. Prawda, S. J. Schlecht, and V. V ̈alim ̈aki, “Robust selection of clean swept-sine measurements in non-stationary noise,” The Journal of the Acoustical Society of America, vol. 151, no. 3, pp. 2117–2126, 2022.
15. [15]D. Fejgin, W. Middelberg, and S. Doclo, “BRUDEX database: Binaural room impulse responses with uniformly distributed external microphones,” in Proc. ITG Conference on Speech Communication, 2023, pp. 126–130.
16. [16]D. D. Carlo, P. Tandeitnik, C. Foy, N. Bertin, A. Deleforge, and S. Gannot, “dEchorate: a calibrated room impulse response dataset for echo-aware signal processing,” EURASIP Journal on Audio, Speech, and Music Processing, 2021.
17. [17]S. V. Amengual Gari, B. Sahin, D. Eddy, and M. Kob, “Open database of spatial room impulse responses at Detmold university of music,” in Audio Engineering Society Convention 149, 2020.
18. [18]“A sonic Palimpsest: Revisiting Chatham historic dockyards.” [Online]. Available: https://research.kent.ac.uk/sonic-palimpsest/impulse-responses/