~/talk

Inside GPTs: Understanding Large Language Models

links:none on file
slides:none on file

Full pipeline from raw web to deployed assistant with concrete numbers: raw filtered Common Crawl becomes 44 terabytes of training text, tokenized into ~50k vocab historically and up to 200,000 tokens for GPT5/6, so English compresses far better than Macedonian (28 vs 60 tokens example); pretraining predicts next-token over trillions of tokens, then supervised fine-tuning with human conversations and RLHF to teach “I don’t know” and prefer answers; techniques include sparse activation (2 billion active of 500 billion), repeating layers to think more, large context windows and agents that execute APIs and manage context.

AI-written summary · about

  1. 0:00Introduction and overview
  2. 1:06Pre-training the model
  3. 5:44Training the neural network
  4. 8:36Inference and token prediction
  5. 10:24Post-training and fine-tuning
  6. 13:18Reinforcement learning from feedback
  7. 16:50Knowledge in parameters vs context
  8. 21:05Training process and thinking steps
  9. 23:56Reinforcement learning details
  10. 25:42How models process language
  11. 27:08Extreme thinking and repetition
  12. 28:33What is an agent
  13. 31:03Agent training and risks
  14. 33:51Future of agents and safety
~/search
type to search 6,518 conferences, 143,918 speakers and 192,133 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
← →
on a talk: 5 seconds back / forward
0–9
on a talk: jump to 0%–90% of the recording
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.