How to Host a (Very) Popular Website for 30 Altairian...
summary
Archive of Our Own serves about 34 billion pages a year (~87–100M pageviews/day) on owned bare metal to avoid deplatforming, runs on Rails with MySQL/MariaDB Galera clusters, MaxScale, ElasticSearch, HAProxy, Nginx, Ansible and FAI, and relies on volunteers — no employees, volunteer-run. Recent costs include buying three heavy DB servers (~$65–67k each) and ~ $550/month cloud spend; a DDoS peaks at ~1.2 Tbps resolved by onboarding Cloudflare Project Galileo and restoring service in ~5.5 hours. PT-Online Schema Change, Redis (noncritical), and local rate-limiting keep the site usable.
AI-written summary · about
chapters (11)
- 0:00Hosting a popular hobby site
- 4:41Building Archive of Our Own
- 10:45Hardware choices and AWS
- 14:36Avoiding sharding, big servers
- 17:03Database scaling with Galera MaxScale
- 20:56Recycling old hardware
- 23:46Reducing complexity with support
- 26:14Traffic shaping with HAProxy
- 30:50Dynamic rate limiting
- 34:22Cloud vs bare metal
- 35:05Q&A
related talks
54mGetting to 300 million requests a month with a one man AI/Dev-Ops team
23mIt Takes a Nation of Millions to Hold Us Back
21mAvoiding That $1M Dollar AWS Bill - Mark Michael, Glenn Oppegard
27mpostgresql.org: The hidden parts
16mThe Wikimedia infrastructure
43mTools and Team for 20 Billion PageView/Month
25mFrom PHP and on-prem servers to NodeJS/React and AWS
52mLibrary of the Future