Modeling Aesthetic Identity- Building a Digital Twin
summary
A reproducible pipeline called kite architecture converts Instagram likes, followed brands and ignored posts into OpenCLIP visual embeddings plus categorical signal embeddings, then conditions a language model via LoRA fine-tuning to predict engagement and produce attention maps and explanations. It trains a 7B quan base with adapters in ~3.5 hours, reports ~78.9% AUC, requires ~100+ liked images and shows early overfitting, drift and privacy limits.
AI-written summary · about
chapters (12)
- 0:00Digital twin project Kite
- 3:14Instagram data and buying behavior
- 7:03Formulating taste from signals
- 11:36Kite architecture and training
- 16:56LoRA adapters for small models
- 19:25Demo: minimalist streetwear persona
- 28:02Training data challenges and metrics
- 33:38Future work and heat maps
- 34:44Limitations and takeaways
- 36:08Brands and Instagram aesthetics
- 40:01Reverse engineering preferences
- 41:48Q&A
related talks
26mMaximizing Multimodal: Exploring the search frontier of text-to-image models
19mJourney to 1000 Models: Scaling Instagram's algorithm without the Reliability Nightmare
52mThe Yoga of Image Generation with Stable DIffusion and ComfyUI
18mInstagram in focus
20mProductionizing On-device AI Models
42mAn Algorithm of Style
13mInside Multimodal Models
05mFine-tuned SigLIP Image