Ethical Human-in-the-Loop ML Pipelines
summary
Argues that ethical human-in-the-loop ML is practical and profitable by giving annotators fair pay, regular payment, transparent ownership, and by building production teams not ad hoc crowds. Recommends active learning with uncertainty sampling and diversity sampling, keyboard-driven annotation UIs, and reuse of Photoshop-like tools. Cites a provider with ~2,800 full-time annotators, <5% attrition, 24.1 average age, 50% women, 70% computer vision work, SOC2 and ISO 27001 security, and custom Panoptix and 3D LiDAR tooling to reach enterprise quality.
AI-written summary · about
chapters (12)
- 0:00Start
- 0:52Starting Your Data Strategy
- 3:18Ensure that Expected Responses Are Diverse To Avoid Priming
- 4:48Keyboard Driven Inputs
- 5:21Knowledge Quadrant
- 6:21Diversity Sampling
- 7:35Uncertainty Sampling
- 11:53Security
- 19:13Importance of Consistency and Quality
- 23:16Retain Learnings across the Iterations
- 23:54Three Kinds of Workforces
- 27:37Building a Mini Accenture for Ai Services
related talks
18mPositively Impacting Communities AND Succeeding as a Woman in Tech: How You Can Aim for Both
26mBuilding ethical AI: Transparency, bias mitigation, and trust
51mKeynote Panel: Open Innovation in AI - Present an...
17mYour AI Deserves Better Data
43mReducing Bias in AI with Open Source- 43mReducing Bias in AI with Open Source
19mMoral Outsourcing: Humanity in the age of AI
01h 26mShipping AI That Works: An Evaluation Framework for PMs