The SmugMug Tale
summary
SmugMug runs photo/video storage on Amazon S3 (measured in petabytes) and EC2 for encoding, with a custom Skynet autoscaler that spins hundreds to thousands of cores on demand. MySQL is the critical store: large Terabytes of InnoDB on 8‑core, 64GB nodes with Percona/1.03 patches, replication tuned to keep slave lag <1s and use memcached to reach a 96% memcached hit rate (reads often 70µs). Using a Toro 7410 SSD appliance over NFS cuts synced‑write latency (10k IOPS ≈700µs) and makes snapshot clones take seconds instead of hours.
AI-written summary · about
chapters (26)
- 0:00Intro
- 1:29Our challenge
- 2:41Architecture overview
- 4:12Storage
- 5:25Compute
- 7:21Secret Weapon:Akamai Super-fast CDN
- 9:19Secret Weapon: memcached
- 10:46Secret Weapon: MySQL
- 13:27Secret Weapon: InnoDB
- 14:45Secret Weapon: Percona
- 17:34MySQL query details
- 19:21MySQL Issues: Filesystems
- 20:47Filesystem Solution - ZFS! Transactional
- 21:52The REAL Issue We run Linux
- 23:40Replication solutions
- 24:49Secret Weapon: Sushi Toro aka S7410.
- 27:13Sushi's quite reasonable
- 28:13Sushi served fast
- 30:37Sushi today
- 31:35Analyzing sushi DTrace on Linux!
- 32:18MySQL on Toro so far NFSv3 (rather than v4)
- 33:24MySQL on Toro problems
- 33:55Even faster?
- 34:27Kitchen sink on Toro
- 34:40The future? 100% SSD
- 35:26Oracle wishlist
related talks
10mTwo Years Later, Loving and Hati
50mDeveloping Highly Scalable Image Storage Solution with AWS Serverless
17mPurikura culture in Japan and our web app architecture
01h 04mHow do you store your data?
08mPivot
25mData in the Cloud
39mHow we built a successful cloud-scale product with GCP
25mGreenOps at Scale: The 50TB Memory Diet for a Petabyte Platform