Now smile (also in 3D)! Exploring AR, ML and Camera related APIs on Android
summary
Recommends using CameraX for photo and video (CameraX 1.3.0 is now stable with effects and video), warns about permissions and bind/unbind lifecycle pitfalls, and shows a working demo that pins ML labels into 3D space using ARCore + ML Kit + TensorFlow Lite; the demo runs both a 170MB local TFLite model and a remote Cloud Vision model (remote requires enabling billing). Practical tips cover image capture outputs, rotation/aspect handling, and error cases.
AI-written summary · about
chapters (7)
- 0:00Introduction and speaker background
- 3:59Camera APIs overview and Polaroid context
- 6:08Basic camera usage on Android
- 7:12CameraX vs Camera2 APIs
- 16:56ARCore and machine learning tools
- 19:23Demo: ARCore ML Kit integration
- 36:15Resources and closing remarks
related talks
44mNow smile (also in 3D)! Exploring AR, ML and Camera related APIs on Android
21mLive Coding: Adding Machine Learning to Android Things in 10 Mins
41mFusing AI with AR: Custom Markers
35mAugmenting Faces and Images
39mA Real use case of practical AR-AI couple: Size my luggage
18mDeep-dive on ARCore | Fundamentals of designing AR apps with machine learning models
37mExploring Augmented Reality with ARCore
30mMachine Learning in Android Apps with ML Kit