From Issue to PR Merged: A Fluentd “Tail”
Talkat KubeCon + CloudNativeCon North America 2019 · on Tuesday, 2019-11-19· at 14:25· Room 7AB - San Diego Convention Center, Upper Level·
summary
Fluentd S3 input plugin gains an optional parameter to read cross-account SQS queue URLs so CloudTrail objects from hundreds of AWS accounts land correctly in a shared bucket; fixes include setting the S3 region and passing queue_owner_aws_account_id to the SQS client, updating mock tests, surviving Travis CI, and landing as a merged PR (about eight lines of Ruby changed). Result: CloudTrail ingestion works across accounts, ~5‑minute delivery, and the pipeline scales by adding Fluentd instances to drain millions of visible SQS messages.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:05Jordan Hamill
- 0:53About Amgen
- 16:06Travis Ci Tests
- 18:04Pull Request Summary
- 19:51Why this Fix Matters
- 27:37Plug-In Ecosystem
- 30:11Demo
- 33:37Clinical Trials
related talks
32mFluent Bit Open Source Project & AWS
36mFluentd Project Intro – Eduardo Silva & Masahiro Nakagawa, Treasure Data
42mIngestion at Scale: Open Source Pipelines for Getting Data Into Open...
10mDrive positive impact with feature flags and data (PRT053)
19mStreamline operational troubleshooting with Amazon Q Developer | AWS Developer Day 2025
36mFrom massive pull requests to trunk-based development with Ruby
22mFrom Bug To PR: Turning Customer Pain Points Into Upstream Contributions
10mAWS SIG Update
