Using Pytorch, AI, WebGL and React to make 3d avatars of our users
summary
Runs a daily cron that sends new user photos to a PyTorch depth model (Midas, later switched to a better “zoo” variant) and stores generated depth maps on CDN, then renders them client-side with a rewritten shader in a WebGL+React component that gracefully falls back to the original image. Replaces painful GPU VPS + Python stack by calling Replicate’s hosted models via API so no maintainer-managed ML infra is needed. Works at scale but Midas has occasional failures; shader fix removed a visible echo.
AI-written summary · about
chapters (7)
- 0:00Introduction and background
- 3:56Why upgrade avatars to 3D
- 11:02Depth map extraction with PyTorch
- 16:27Solving scalability with Replicate
- 22:38Displaying 3D avatars in React
- 24:46Future of AI-driven 3D avatars
- 26:54Conclusion and takeaways
related talks
31mNon-traditional usage of Cloud for Science Art
18mBuilding Efficient PyTorch Models on Vertex AI
40mAAA 3D graphics on the Web with ReactJS + BabylonJS + Unity3D
32mSelf-Portraits in JavaScript
20m3D and Canvas-Based Animations with Reanimated
11mLaunch by Reface
30mBuilding Machine Learning Models on Edge with React
58mPushing Deepfakes to the Limit-Fake video calls with AI