Is This Feature Interesting? Interfeat: AI for Insight Discovery
summary
InterFeat is a modular pipeline that filters thousands of tabular features by utility filters (p‑values, mutual information, ML importance), novelty filters (SemMedDB/knowledge‑graph + PubMed cooccurrence) and an LLM annotation layer (RAG/BM25 for evidence). Applied to the UK Biobank (~400,000), it narrows candidates to on the order of 50 per disease, recovers about 20% discovered years earlier in literature, and yields higher expert‑validated interestingness than SHAP baselines; code on github.com/ddofer/InterFeat.
AI-written summary · about
chapters (9)
- 0:00Finding interesting features
- 3:11Why features matter
- 8:28What makes a feature interesting
- 10:32Interfeat pipeline overview
- 16:30Filtering by novelty
- 19:49LLMs for plausibility
- 21:57Evaluating with experts
- 25:47Key findings
- 26:50Conclusion
related talks
34mFrom Data to Discovery: How AI Is Finding Hidden Cures
22mOne-click analysis: Detecting and visualizing insights automatically
36mFormatting Biological Big Data to Enable (Personalized) Systems Pharmacology
20mDiagnostic Medical application of ML
01h 28mFeature Engineering Made Simple
10mDeepSI: Interactive Deep Learning for Semantic Interaction
24mHybrid ML Methods for the Interpretation and Integration of Heterogeneous Multimodal Data
10mEmpowering your feature stores with AI feature discovery