DataEngBytes 2023 - MEL-T1-11 - Emily Melhuish

on YouTube ↗28m

" summary

Builds datasets from raw sources: parses kHz engine sensor logs on‑prem with vanilla Python, stores in Postgres/TimescaleDB and computes flight parameters used on Electron launch; ingests collar GPS/IMU at ~10s Hz with Kinesis, AWS Glue (PySpark), Athena and hires cow behavioralists labeled grazing by video for training data; pulls GitHub/Jira events, processes with Lambdas, Step Functions, ECS, dbt and Snowflake, serves results from Postgres and just-launched AI coach via Slack. Emphasizes automation, schema, security and ethics.

AI-written summary · about

" chapters (7)

1. 0:00Introduction
2. 4:18Rocket engine testing pipelines
3. 6:49Cow behavior prediction
4. 11:05Team performance tracking
5. 14:18Five lessons from startups
6. 23:27Data ethics and security
7. 26:15Summary

" related talks

• 21mDataEngBytes 2023 - SYD-T2-04 - Suneeta MallSuneeta Mall · DataEngBytes Sydney 2023
• 21mDataEngBytes 2023 - SYD-T2-02 - Vinny Vijekumaar & Zak SheikhVinny Vijekumaar, Zak Sheikh · DataEngBytes Sydney 2023
• 20mDataEngBytes 2023 PER-T1-10 - Matthew CarterMatthew Carter · DataEngBytes Perth 2023
• 21mDataEngBytes 2023 - SYD-T1-06 - Sana SanaiSana Sanai · DataEngBytes Sydney 2023
• 28mDataEngBytes 2023 - PER-T1-04 - Nathan GloverNathan Glover · DataEngBytes Perth 2023
• 44mBest practices for analytics and generative AI on AWS(ANT302)Magdalena Gargas, Nat Noordanus · AWS Summit Zurich 2024
• 29mDataEngBytes 2023 - SYD-T1-08 - Rishu Saxena & John CosgroveRishu Saxena, John Cosgrove · DataEngBytes Sydney 2023
• 22mDataEngBytes 2023 - BRI-T1-06 - Yanir SeroussiYanir Seroussi · DataEngBytes Brisbane 2023