The Sound of Your Secrets: Teaching Your Model to Spy So You Can Learn to Defend
summary
Demonstrates a working acoustic side‑channel pipeline that converts WAV files to spectrogram PNGs, trains a CotesNet‑style CNN and exports a TensorFlow SavedModel; a single‑keyboard model achieves near 100% accuracy, a multi‑keyboard model scores 46% top‑1 on 38 classes (random = ~3%), and a live demo reconstructs “hello Paris” by combining top‑K candidates, spell‑checking and a masked language model. Training on an H200 takes ~14 hours. Covers practical countermeasures: signal masking, password entropy, environmental noise, and hardware 2FA.
AI-written summary · about
chapters (14)
- 0:00Acoustic side channel attacks
- 1:47AI and ML moral duality
- 8:02Acoustic keystroke logger
- 12:16Training model with keyboard data
- 16:23Multiple keyboards and real-world risks
- 20:13Decoding keystroke messages
- 27:22Automating word prediction
- 30:57Single vs multiple keyboard accuracy
- 33:47Attack improvement with manufacturing similarity
- 36:35Attack vectors and defense strategies
- 38:44Protecting from acoustic attacks
- 40:31Facial movement threats
- 41:13Resources and further exploration
- 41:56Q&A
related talks
01h 14mExploring the hidden potential of sound data
15mMaking Acoustic Side-Channel Attacks on Noisy Keyboards Viable with LLM-Assisted...
28mHow to use Deep Learning to process audio data and classify sounds.
32mAfter Censorship, Self-Censorship… but This Time It’s Funny, ...
20mExploring the hidden potential of sound data
30mSound Modelling - parametric methods and deep learning...
37mKeynote - Hacking AI How to Survive the AI Uprising
25mLooking at Sound