From custom Solr to out-of-the-box OpenSearch
summary
Long-running Solr/Lucene code and in-process indexing cause versioning and scalability pain, so the stack moves from embedded Solr to Elasticsearch and then OpenSearch by replacing polling every few seconds with an event-driven Kafka pipeline that updates indexes near real time. Metadata gets indexed immediately while heavy document text extraction is handled asynchronously so content appears later. Migration uses parallel reindexing from the database to avoid downtime; if source access is unavailable options include Spark reads or dumping Parquet files and ingesting via Data Prepper.
AI-written summary · about
chapters (6)
- 0:00Migration journey
- 1:08Legacy Solr flaws
- 3:41OpenSearch architecture
- 5:54Document indexing challenges
- 6:14Containerized deployments
- 7:00Q&A
related talks
40mDTEX's OpenSearch Journey: From Migration to Innovation
25mRunning OpenSearch in the Real World: Migration, Mistakes and Maturity
42mOpenSearch Migrations: Modernize Without Worry
21mWorking in the Open...Search
36mWhat’s New in the OpenSearch Project and Ecosystem
40mInnovating in Open Source in Your Enterprise
42mIngestion at Scale: Open Source Pipelines for Getting Data Into Open...
12mKeynote: OpenSearch, Innovation and Community