RealTime Exoskeleton-Mapping and Body Segmentation on the Browser with Tensorflow.js
summary
Shows a runnable vanilla JavaScript pipeline that loads TensorFlow.js and BodyPix via CDN, runs webcam inference in the browser, and outputs pixel-accurate poses and part masks (keypoints given in pixels). Chooses MobileNet or ResNet, configures multiplier, internalResolution and quantBytes, and handles multi-person detection with maxDetections. Demonstrates blurring left/right face when a hand overlaps the head using part IDs 0,1,10,11 and a 10% intersection threshold, uses an async request loop and canvas drawing — real-time and working.
AI-written summary · about
chapters (21)
- 0:00Introduction and speaker intro
- 1:02What is TensorFlow.js and BodyPix
- 2:27Setting up the project
- 3:08Loading the BodyPix model
- 5:19Segmenting a single person
- 8:07Segmenting multiple people
- 11:20Visualizing segmentation results
- 13:27Segmenting body parts
- 14:31Applying blur effects
- 16:20Blurring specific body parts
- 17:24Live demo setup
- 18:30JavaScript implementation details
- 21:09Detecting hand in front of face
- 27:16Using key points for applications
- 33:39Final results and use cases
- 34:42Q&A session introduction
- 35:24Comparison with other ML libraries
- 41:52Learning curve and documentation
- 43:38Suggested beginner projects
- 46:53Practical applications at home
- 50:07Closing remarks and Q&A wrap-up
related talks
49mReal Time Face Detection through a Webcam Using AI
41mTensorFlow.js 101: ML in the Browser and Beyond
54mDeep Learning in JS
29mSuperpowers for next gen web apps: Machine learning
15mMachine learning for next gen web apps with TensorFlow.js
20mTensorFlow.js Bringing Machine Learning to the Web and Beyond
30mBuilding Machine Learning Models on Edge with React
32mSelf-Portraits in JavaScript