~/talk

On the path to a massive scale SolrCloud: removing Overseer

on YouTube 43m
Talkat ApacheCon@Home 2021 · 2021-09-21..09-23· [distributed-systems]

speaker

name:Ilan Ginzburg
links:none on file

resources

video:https://youtu.be/ofTfsOZHd3Y
slides:none on file

summary

SolrCloud removes the single Overseer bottleneck by making nodes update cluster state directly in Zookeeper using compare-and-swap (CAS) for state.json and coordinating long-running Collection/Config Set commands with Zookeeper-based distributed locks. Asynchronous Collection API semantics are preserved via a mix of persistent and ephemeral znodes. Distributed mode is opt-in today and can be enabled in Solr 9; most Collection API operations become faster, while high-contention concurrent state mutations may incur retries.

AI-written summary · about

chapters (10)

  1. 0:00Introduction and motivation
  2. 2:22Understanding SolrCloud architecture
  3. 6:37Limitations of the Overseer
  4. 10:55Proposed distributed architecture
  5. 15:19Implementing state changes
  6. 18:38Distributed locking strategy
  7. 22:26Asynchronous API execution
  8. 25:49Benefits and deployment plan
  9. 30:37Future scaling work
  10. 35:42Q&A session

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.