Lost in Transliteration: Why strlen("Dvořák") Returns 8
summary
An exploration of character encodings and how C libraries handle them, starting with ASCII, code pages, and Unicode and then focusing on the C runtime behavior. It examines iconv and glibc’s gconv pipeline, explains why functions like strlen can give unexpected results (e.g., strlen("Dvořák") == 8) and why conversion flags like //IGNORE are inconsistent, and presents practical insights from the speaker’s experience as a glibc contributor.
AI-written summary · about
chapters (18)
- 0:00Text encoding confusion
- 1:47strlen returns 8
- 2:58ASCII vs code pages
- 6:07Multibyte encoding issues
- 6:30Unicode solution
- 8:13Code points vs storage
- 10:24UTF-8 UTF-16 UTF-32
- 11:28C strings and bytes
- 13:57Wide chars and locales
- 15:23Encoding conversion
- 18:18Conversion fails
- 20:02Text processing impact
- 21:05Encodings needed
- 22:50ICU pipelines
- 23:54Encoding in 2026
- 25:17Turkish eye bug
- 26:23Unicode across languages
- 28:28Q&A
related talks
01mTalk Preview: How Symbols Work and Why We Need Them
59m“C++ in Elvenland”
01h 02mHow symbols work and why we need them
01h 02mHow Linux Elf Symbols Work and How They Are Used in C++ and C Programming
01h 26mLinux User/Kernel ABI: the realities of how C and C++ programs really talk to the OS
01h 04mThe Bits Between the Bits: How We Get to main()
01h 00mFrom Program to Process - What Happens After the Compiler
55mSorting Out glibc Collation Challenges