Fine-tuning Neural Sparse Model for Domain Specific Data From an Existing OpenSearch Index
summary
Fine-tunes sparse encoders using an OpenSearch index, Hugging Face/BERT variants and open-source tooling, exporting models as TorchScript and deploying them inside OpenSearch. Training uses an Ollama 2.57B LLM to synthesize queries and hard/soft negatives from 49 documents, runs on a single CPU node (8 cores, 16 GB) and completes in about 90 minutes training. Results move NDCG from ~0.71 (base mini) to ~0.76 after full re‑training, with prior sparse‑head‑only runs reaching ~0.89; PubMed BERT improves vocabulary coverage but needs more pretraining to beat base in some cases.
AI-written summary · about
chapters (17)
- 0:00Fine-tuning neural sparse models
- 1:26NeuralSparse search overview
- 3:33Pre-trained sparse model limits
- 6:01Why fine-tune domain models
- 6:43Sparse encoder components
- 10:16Training setup and base models
- 14:13Fine-tuning pipeline and data prep
- 15:18Demo: fine-tuning process
- 20:46Generating training data with LLM/OpenSearch
- 22:32Training with in-batch negatives
- 25:20Training duration and hardware
- 26:51Probing and validating model
- 31:00Exporting model to OpenSearch
- 32:46Side-by-side model comparison
- 34:53Evaluating search relevance
- 37:46Key takeaways
- 39:52Resources and closing
related talks
41mFine-tuning Neural Sparse Model for Domain Specific Data from...
32mBudget Friendly Semantic Search With Neural Sparse Search
13mFast and Explainable Search on a Budget With OpenSearch Neural Sparse
45mRevolutionizing Search with Power of Neural Sparse and Dense Vectors
44mRediscover your keyword search: Expand, Enrich and Rewrite
31mIntelligent Japanese Search With OpenSearch
39mModel Fine-tuning For Search: From Algorithms to Infra
09mFine-tuning embeddings 4 vector search