An FFmpeg maintainer's deep dive: iOS VideoToolbox and Android MediaCodec integration

on YouTube ↗11m

" summary

FFmpeg exposes iOS VideoToolbox and Android MediaCodec so hardware H.264 decoding returns platform-specific frames instead of raw YUV buffers, enabling a zero-copy rendering pipeline into CVPixelBuffer-backed textures. The repo contains a VideoToolbox device context that hands back CVPixelBuffer frames and a MediaCodec path that can attach an Android Surface and produce a media codec pixel format. iOS path is mature and synchronous; Android path is more complex and still evolving but functional for decoding and rendering.

AI-written summary · about

" chapters (8)

1. 0:00Introduction to ffmpeg
2. 1:34Core ffmpeg libraries
3. 2:51Hardware acceleration
4. 3:35iOS video toolbox
5. 5:12Rendering iOS frames
6. 7:58Android media codec
7. 9:11Android rendering
8. 9:53Conclusion and call to action

" related talks

• 01h 01mA Deep Dive into 2 Cross-Platform Mobile Apps Written in C++T. Grue, S. Kabbes · CppCon 2014
• 41mCrossplatform 2019: C ++ as an antagonist in a progressive worldAleksandr Efremenkov · DevFest Gorky 2019
• 33mThe (not so) hidden cost of sharing code between iOS and AndroidEyal Guthmann · droidcon SF 2019
• 29mWorking with C and C++ in Apple environmentRichard Topchii · #Pragma Conference 2023
• 57mContemporary C++ in ActionDaniela Engert · C++ on Sea 2022
• 01h 03mContemporary C++ in actionDaniela Engert · NDC TechTown 2022
• 01h 20mAudio in standard C++Timur Doumler · ACCU 2019
• 45mHow to combine Swift/Kotlin with C/C++Michał Kowalczyk · Mobilization 2017