Using Sockeye Neural Machine Translation in a Streaming Pipeline
summary
A concrete pipeline that consumes live tweets, detects language with Apache OpenNLP, sends sentences via Apache Flink and Thrift to a Sokka neural model and writes translations back to the console. Claims model sizes drop from tens of gigabytes for statistical systems to ~200MB for Sokka, training runs take days on GPUs but inference runs realtime on a small t2 instance, and byte pair encoding reduces unknown words. Demo translates German tweets to English successfully.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:04Machine translation importance
- 3:27Statistical MT overview
- 7:20Statistical MT challenges
- 9:27Neural MT introduction
- 12:18Switching to neural MT
- 16:55Sockeye vs Joshua
- 19:47Unknown words with BPE
- 21:56Sockeye toolkit
- 24:46Flink for streaming
- 26:12Streaming pipeline example
- 29:36Real-time demo
- 31:01Open-source credits
- 32:34Q&A
related talks
41mAdding Generative AI to Real Time Streaming Pipelines
43mSentiment Analysis in Action: Building Your Real-time Pipeline
31mThe Fast and the Fluent: AI-Powered Speech Translation at the Edge With K0s
21mFrom 0 to a crowd sourced neural network in a jiffy
01h 25mApache Spark for Big Data Processing
52mMachine Learning Data Pipelines
45mTranscend Language Barriers with Custom Machine Learning APIs: um...
50mThe tech behind modern AI/ML use cases in Gaming