~/talk

DataEngBytes 2023 - MEL-T3-02 - Alexei de Lauw

on YouTube 18m
Talkat DataEngBytes Melbourne 2023 · on Thursday, 2023-08-31· [databases, data-engineering]

speaker

name:Alexei de Lauw
links:none on file

resources

video:https://youtu.be/Dxjms4fUrLQ
slides:none on file

summary

Demonstrates logging and tracing built on Snowflake event table primitives to solve three concrete bottlenecks: debugging long stored procedures and UDFs, avoiding table concurrency locks from many writers, and mitigating micro‑partition fragmentation from single‑row inserts. Implements via UDFs/UDTFs and stored procedures in SQL/Python/Java/JS, uses serverless compute that auto‑scales, shows cost tracking in account_usage.event_usage_history, charges about 1.25 credits for serverless plus cloud services, and enforces spans limited to 128 attributes and 128 events; demo runs and writes real trace and log rows.

AI-written summary · about

chapters (13)

  1. 0:00Introduction
  2. 3:09Underdog features
  3. 4:12Snowflake logging bottlenecks
  4. 6:39Logging and tracing solution
  5. 9:07Logging and tracing setup
  6. 9:50Logging and tracing costs
  7. 11:17Demo setup script
  8. 13:04Python logging procedure
  9. 13:48Python tracing with spans
  10. 15:13Querying event logs
  11. 15:59Viewing tracing data
  12. 16:40Monitoring Snowflake credits
  13. 17:05Q&A

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.