Making Acquaintances with Exotic Encodings
summary
Claims that Unicode isn’t universal: many sites and emails still use legacy sets like ISO-8859-1, Windows-1252, Shift_JIS, Big5 or GB2312, while UTF-8 dominates (~93% of websites). Python techniques include using the codecs.encode/decode APIs with error handlers, heuristics from chardet as a last resort, and a maintained static fallback table of observed encodings (company corpus). Empiric numbers: declared encodings succeed ~75%, chardet resolves most of the rest (~98%), and the static table catches the stubborn few.
AI-written summary · about
chapters (5)
related talks
40mUnicode: The hero or villain?
01h 24mUnicode: From First Principles to Practical Implementation
32mHow to Become an Encoding Champion
48mUnicode: Going Down the Rabbit Hole
45mI ♥ Unicode
54mSTRINGS ATTACHED: FROM ASCII TO 🙊🙈🙉
33mPython's Transition to Multilingual Programming
11mLightning Talk: Encoding - Understanding the Basics