GPT-Whale Language Models at Tools for Structure Discovery and Hypothesis Generation
summary
Uses language-model black boxes trained on annotated sperm whale codas (about 3,840 codas) represented as discrete tokens (roughly 21–26 coda types) and compares n-gram, RNN, LSTM and small transformer/GPT-style models. Ablation experiments—5-coda context truncation, ablations: shuffle/drop clicks and tempo/rhythm removals—show that modest long-range context improves next-coda and dive-timing prediction, that extra clicks influence chorusing decisions, and that models trained on limited data are noisy but reliably expose which signal features carry usable information for hypotheses and playback experiments.
AI-written summary · about
chapters (7)
- 0:00Language models as black boxes
- 4:23Defining the black-box approach
- 6:45Methodology for model analysis
- 12:32Decoding human brain activity
- 21:56Sperm whale communication
- 30:01Whale communication analysis
- 40:36Q&A and future directions
related talks
53mWhAM: Towards A Translative Model of Sperm Whale Vocalization
53mAI as a tool for discovery in animal communication
01h 07mDecoding Dolphin Speech
01h 00mLong term work and Machine Learning Approaches to DolphinGemma
21mWhale: Efficient Giant Model Training over Heterogeneous GPUs
46mUnder the Hood of LLMs — all those questions you never dared ask about them
01h 00mInside GPT – Large Language Models Demystified
01h 38mTraining Large Language Models: Practices and Research Questions