Introducing VoicePrivacy

The VoicePrivacy initiative is spearheading the effort to develop privacy preservation solutions for speech technology. It aims to gather a new community to define the task and metrics and to benchmark initial solutions using common datasets, protocols and metrics. VoicePrivacy takes the form of a competitive challenge. The challenge is to develop anonymization solutions which suppress personally identifiable information contained within speech signals. At the same time, solutions should preserve linguistic content and speech quality/naturalness. The challenge will conclude with a session/event held in conjunction with Interspeech 2020 at which challenge results will be made publicly available.

Subscribe

Participants are encouraged to subscribe to the VoicePrivacy 2020 mailing list by sending an email to:

sympa@lists.voiceprivacychallenge.org

with “subscribe 2020” as the subject line. Successful registrations are confirmed by return email.

To post messages to the mailing list itself, emails should be addressed to:

2020@lists.voiceprivacychallenge.org

Register

Participants are requested to register for the evaluation. Registration should be performed once only for each participating entity and by sending an email to:

sympa@lists.voiceprivacychallenge.org

with “VoicePrivacy 2020 registration” as the subject line.

The mail body should include:

(i) the name of the team; (ii) the name of the contact person; (iii) their country; (iv) their status (academic/nonacademic).

Schedule

Release of evaluation plan 6th February 2020
Release of training and development data 8th February 2020
Release of evaluation data 15th February 2020
Deadline-1 for participants to submit objective evaluation results 8th May 2020
Interspeech-2020 paper submission deadline 8th May 2020
Deadline-2 for participants to submit evaluation results and data 16th June 2020
Submission of system descriptions 23rd June 2020
Submission of additional data (optional) 1st July 2020
Organizers return subjective evaluation results to participants Early/mid August 2020
VoicePrivacy special session/event at Interspeech 2020 26th–29th October 2020
Journal special issue submission deadline Early 2021

Participants may submit to one or both deadlines. Interspeech paper submission is encouraged but optional. All participants will be invited to present their work at the VoicePrivacy session/event.

Data

Several publicly available corpora will be used for training, development and evaluation of speaker anonymization systems. The detailed development and evalaution subsets are described in Evaluation plan. They will be comprised of subsets from the following corpora:

Training

  1. VoxCeleb-1,2
  2. Librispeech (train-clean-100, train-other-500)
  3. LibriTTS (train-clean-100, train-other-500)

Development

  1. Librispeech subset libri_dev can be downloaded from server in run.sh
  2. VCTK subset vctk_dev can be downloaded from server in run.sh

Baselines

We provide a software for two different anonymization system baselines:

  1. Baseline-1: Anonymization using x-vectors and neural waveform models
  2. Baseline-2: Anonymization using McAdams coefficient

https://github.com/Voice-Privacy-Challenge/Voice-Privacy-Challenge-2020

Samples:

The following are examples of original and anonymised versions (primary Baseline-1).

  • LibriSpeech utterances:
original
anonymised
female
male


  • VCTK utterances:
original
anonymised
female
male

Evaluation metrics

We propose to use both objective and subjective metrics to assess speaker verification/re-identification ability. In addition to preserving privacy, the privacy-driven transformation should preserve speech intelligibility and naturalness when used in human communication scenarios, and automatic speech recognition (ASR) training and/or testing performance when used in human-machine communication scenarios.

Objective measures

  1. Speaker verifiability:
    • Equal error rate (EER)
    • Log-likelihood-ratio cost function Cllr / minCllr
  2. Speech intelligibility:
    • Word error rate (WER)

Subjective measures

  1. Subjective speaker verifiability
  2. Subjective speaker linkability
  3. Subjective speech intelligibility
  4. Subjective speech naturalness

Organisers (in alphabetical order)

Jean-François Bonastre - University of Avignon - LIA, France
Nicholas Evans - EURECOM, France
Fuming Fang - NII, Japan
Andreas Nautsch - EURECOM, France
Paul-Gauthier Noé - University of Avignon - LIA, France
Jose Patino - EURECOM, France
Md Sahidullah - Inria, France
Brij Mohan Lal Srivastava - Inria, France
Natalia Tomashenko - University of Avignon - LIA, France
Massimiliano Todisco - EURECOM, France
Emmanuel Vincent - Inria, France
Xin Wang - NII, Japan
Junichi Yamagishi - NII, Japan and University of Edinburgh, UK

Acknowledgements

This work was supported in part by the French National Research Agency under projects HARPOCRATES (ANR-19-DATA-0008) and DEEP-PRIVACY (ANR-18-CE23-0018), by the European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No. 825081 COMPRISE, and jointly by the French National Research Agency and the Japan Science and Technology Agency under project VoicePersonae.

ANR
comprise
EC
JST