GenAI synthetic data used to fine tune a domain specific model
summary
Fine-tunes an open‑source CLIP (ViT‑B/32) model (≈151M parameters) with synthetic labels generated by GPT‑4 Vision, using ~5k London street‑view image–text pairs (noting a small subset of ~400 images can already work). Training runs three epochs on a local M1 MacBook; top‑1 retrieval jumps from about 0.53 to 0.89 (MRR/top‑1), training accuracy ≈0.9. Practical outcome: a lightweight, self‑hosted multimodal search model that finds shops and bike parking much better and runs far cheaper than calling large closed models.
AI-written summary · about
chapters (9)
- 0:00Project introduction
- 1:27General purpose limits
- 4:27Closed vs open models
- 6:20CLIP architecture
- 11:23Fine-tuning CLIP
- 14:15Synthetic labels with GPT-4
- 16:24Fine-tuning process
- 17:30Model results comparison
- 18:54Key takeaways
related talks
33mBuild Better AI with First-Party Data and Fine-Tuning
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
39mCollect Data to Fine-Tune Your Large Language Model
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
30m🤯 No data? No problem! Synthetic data to the rescue
39mNew multimodal vision AI models and their practical applications
46mAttention to Detail: Fine-Grained Vision-Language Alignment
02h 50mUnderstanding and Using AI Language Models