Managing and Monitoring Marklogic for Reliability At Scale: A Customer Use Case
summary
Runbooks and configs that let MarkLogic handle billions of records and thousands of transactions per second: use production‑scale, redacted data for load tests; separate enodes and dnodes to autoscale front ends and protect data nodes; enable debug logs and support dumps; throttle background IO and tune cache partitions and in‑memory stand sizes; prefer range indexes for high‑volume lookups; set strict distributed timestamps and align client/load‑balancer timeouts; use the fast data directory and dynamic enodes where available.
AI-written summary · about
chapters (27)
- 0:00Introduction and speaker background
- 1:15Defining success and system goals
- 3:19Performance testing at scale
- 6:56Testing failure and resilience
- 8:23Tuning and scaling strategies
- 10:08Cluster architecture and node roles
- 11:59Cache and group settings optimization
- 13:44Background IO and logging levels
- 15:55Database and forest organization
- 17:21In-memory settings and index detection
- 19:05Storage failure handling
- 20:30Application server and app threads
- 21:55Distribute timestamp and timeouts
- 23:18Authentication and connection pools
- 24:22Load balancer and health checks
- 25:24Timeout alignment and client behavior
- 26:50High availability and scaling options
- 28:38Resource utilization and HA scenarios
- 30:01Scaling up versus scaling out
- 31:46Cloud and hardware considerations
- 32:27Monitoring and alerting fundamentals
- 34:09Key system metrics to watch
- 37:45Request queuing and slow queries
- 38:27Request logs and query profiling
- 39:55Automatic failure detection
- 42:01Operational maturity and SOPs
- 43:26Resources and closing remarks
related talks
30mCentralized Log Management
40mBuilding Observability with Elasticsearch: Practices for High Load
59mMonitoring A-Z
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
30mData @Scale Seattle- Sundaram Narayanan
11mBuilding a system that never stops: New Relic at scale
44mPlanning for Overload
11mLightning Talk: Beyond Billions: Operating Thanos, Prometheus & OpenTelemetry...