Long-Term Conversation Analysis: Privacy-Utility Trade-off under Noise and Reverberation

Recordings in everyday life require privacy preservation of the speech content and speaker identity. This contribution explores the influence of noise and reverberation on the trade-off between privacy and utility for low-cost privacy-preserving methods feasible for edge computing. These methods compromise spectral and temporal smoothing, speaker anonymization using the McAdams coefficient, sampling with a very low sampling rate, and combinations. Privacy is assessed by automatic speech and speaker recognition, while our utility considers voice activity detection and speaker diarization. Overall, our evaluation shows that additional noise degrades the performance of all models more than reverberation. This degradation corresponds to enhanced speech privacy, while utility is less deteriorated for some methods.

💡 Research Summary

This paper investigates how environmental noise and reverberation affect the privacy‑utility trade‑off of low‑cost, edge‑computable speech processing techniques. The authors focus on everyday long‑term recordings captured by portable devices and aim to protect two privacy aspects defined by the EU GDPR: the linguistic content of speech and the speaker’s identity. To this end, they evaluate four inexpensive privacy‑preserving methods: (1) spectral smoothing by reducing the number of Mel‑filterbank channels from 80 (baseline) to 10, (2) temporal smoothing where power spectral densities are low‑pass filtered with time constants τ = 125 ms, 250 ms, or 375 ms and then subsampled, (3) speaker anonymization using the McAdams coefficient (randomly sampled between 0.5 and 0.9 per utterance), and (4) low‑frequency audio by down‑sampling to 1.25 kHz and discarding frequencies above 625 Hz. Some experiments combine the McAdams transformation with spectral or temporal smoothing.

Privacy is quantified by two automatic systems: an automatic speech recognition (ASR) model (transformer encoder‑decoder with CTC) measured by word error rate (WER), and an automatic speaker verification (ASV) model (ECAP‑A‑TDNN embeddings with cosine scoring) measured by equal error rate (EER). Higher WER and higher EER indicate stronger privacy protection.

Utility is assessed through a voice activity detection (VAD) model (convolutional‑recurrent network) evaluated with Matthews correlation coefficient (MCC) and a speaker diarization (SD) system that re‑uses the same ECAP‑A‑TDNN embeddings and performs spectral clustering, evaluated with diarization error rate (DER). The goal is to keep VAD and SD performance high while degrading ASR and ASV.

All experiments are conducted on publicly available corpora: LibriSpeech for ASR, VoxCeleb2 for speaker embeddings, the VPC test sets for ASV, the AMI meeting corpus for SD, and LibriParty for VAD. To simulate a “semi‑informed attacker” (the strongest realistic threat), the authors fine‑tune or retrain the ASR, ASV, and VAD models on data that has already been processed with the privacy‑preserving methods. Noise is added from 843 point‑source recordings drawn from the MUSAN corpus at signal‑to‑noise ratios (SNR) of 10 dB, 5 dB, and 0 dB, representing typical real‑world conditions. Reverberation is simulated by convolving test audio with three room impulse responses (RT60 = 0.21 s, 0.37 s, 0.70 s) measured in a meeting room, an office, and a lecture hall.

The results reveal two clear patterns. First, decreasing SNR degrades the performance of all models more severely than increasing reverberation time. ASR WER rises sharply as noise increases, especially for methods that already reduce information (e.g., 10‑filter spectral smoothing or τ = 250 ms temporal smoothing). ASV EER also climbs, indicating that noise makes speaker verification more random. In contrast, VAD MCC remains relatively stable across most conditions, showing robustness to both noise and reverberation, except when McAdams anonymization is applied; in that case, added noise harms VAD considerably because the transformation also processes silent intervals. SD DER generally worsens with lower SNR, but some methods (pure spectral smoothing) are more tolerant, while combinations involving McAdams or low‑frequency audio suffer larger degradations.

Reverberation has a milder impact than noise. For most methods, ASR and ASV performance changes only slightly with longer RT60, and confidence intervals often overlap. However, low‑frequency audio is an exception: its ASR performance drops more with increasing reverberation than with decreasing SNR, likely because the limited bandwidth makes the system more sensitive to temporal smearing introduced by room reflections.

Overall, the study demonstrates that environmental degradation (noise, reverberation) can be leveraged to enhance privacy: higher WER and EER mean the speech content and speaker identity are less recoverable. At the same time, utility (VAD, SD) does not always suffer proportionally; certain low‑cost methods—particularly simple spectral smoothing (10 Mel filters) or short temporal smoothing (τ = 125 ms)—provide a favorable balance, preserving useful voice activity and diarization cues while substantially increasing privacy. The authors conclude that these inexpensive techniques are viable for edge devices that must operate under strict power, memory, and latency constraints while complying with privacy regulations. Future work is suggested to explore alternative feature representations beyond standard Mel filterbanks and to model more realistic acoustic degradations, aiming to jointly optimize privacy and utility for real‑world long‑term conversational analysis.