Mastering Visual AI with Vision-Language Models & Advanced Evaluation Techniques
summary
Hands-on workflow loads the CarDD dataset from Hugging Face into FiftyOne app (2,816 samples, 6,211 damage detections), computes CLIP/SIGLIP/RADIO embeddings, and builds dashboards showing clusters, duplicates, and patch views. Vision‑language models (Fast VLM, Neotron) generate damage reports, camera perspective, color and location fields; representativeness, uniqueness and complexity scores identify edge cases. A fine‑tuned YOLO is applied and evaluated with per‑scenario metrics, confusion matrices, and a multi‑tiered auto‑approve / expert review deployment plan.
AI-written summary · about
chapters (14)
- 0:00Visual AI workshop intro
- 1:54Workshop overview
- 7:51Loading CAR DD dataset
- 14:10Exploring data with FiftyOne
- 20:33Creating data slices
- 21:58Enriching data with scores
- 26:14Computing embeddings
- 33:15Visualizing embeddings
- 34:41Measuring representativeness
- 43:27Enriching with VLMs
- 56:20Fine-tuning model
- 59:12Evaluating with FiftyOne
- 1:02:04Deployment strategies
- 1:06:21Key takeaways
related talks
40mKeynote: Open-source Multimodal AI in the Wild
51mVisual Quality Assurance Using Vision-Language Models
38mHow to Test a learning system
17mVision AI in 2025
23mVision-Language Models for Fraud Detection
01h 00mData, visualization, and designing AI
45mWhat if AI were the solution to understanding sign language?
37mVisual Literacy: Complex Document Retrieval with VLMs