How to Tackle MediaPipe Face Landmarker Accuracy Limits — Practical Landmark Geometry Corrections
summary
Examines shortcomings in MediaPipe Face Landmarker where seven blendshape parameters (cheekPuff, tongueOut, jawForward, jawRight, jawLeft, mouthDimpleRight/Left) stay near zero despite 478 landmarks and 52 blendshapes. Describes a geometry-based solver called BlendShapeEnhancer implemented as a Kotlin Multiplatform SDK that boosts cheek-width sensitivity (×3), encodes FACS co-activation rules, detects all seven puff tests with zero false positives on 11 non-puff trials, and compares this approach to a PyTorch ML model which yields better accuracy but increases model size and latency; field validation is not yet done.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 3:30Face tracking mechanics
- 14:24MediaPipe landmarks
- 17:12Cheek puffing Android
- 23:39Detecting cheek puff
- 27:56Geometric solver
- 31:07SDK design
- 36:46Summary
related talks
21mEmotion Estimation for Social Robotics
40mOn-Device Machine Learning for Android with MediaPipe
32mAdvanced on-device ML made easy
20mA Girls Guide To Growing A Moustache with Python
19mBuilding Machine Learning Applications using Hugging Face
15mC-Face: Continuously reconstructing facial expressions by deep learning contours of the face ...
48mImage Recognition and Object Detection with MediaPipe on A…
27mSign Language Recognition Improvements Since the London Keynote