Computer Vision and NLP for Multi-Task Fashion Attrb. Modeling
summary
Claims an operational pipeline: an ensemble of custom multi-task CNNs and a fine-tuned Google’s Bert in Pytorch that tags millions of products for Shoprunner, giving a few percentage points lift when combined. Image model uses full image plus center crop into two ResNet-50 towers (2048+2048 → 4096) into a dense bottleneck and task heads. Training runs on AWS p3 GPUs, datasets are ~100–200k items each, and a multi-dataset multitask training scheme samples task-specific batches and backpropagates only that task loss. Models serve on CPU and act as feature extractors for search and similarity.
AI-written summary · about
chapters (15)
- 0:00ShopRunner intro and fashion attributes
- 1:07Attribute definition and standards
- 2:31Attribute modeling development
- 3:56Images and text for attributes
- 5:42Multitask learning analogy
- 7:49Hard parameter sharing
- 9:11Training new task heads
- 9:55Image model architecture
- 13:48NLP model with BERT
- 16:58Neural network ensemble
- 18:20Expanding for new attributes
- 19:45Franken model process
- 22:15Multi-dataset training
- 25:50Q3 goals and plans
- 27:14Q&A on training costs
related talks
44mWorking with Fashion Models
36mFinding needles in haystacks with Deep Neural Networks
51mCognition and Computation
33mMachine Learning in Visual Search
42mUsing Interconnected ML Models to Tackle Retail Challenges
28mBuilding one multitask model to rule them all
22mHarnessing image and text data for product matching
25mAI and Fashion- Product Retrieval with Multi-modally Generated Data