An APL Interface to Kafka
summary
A Dyalog interface implements Kafka client functionality in APL and C using librdkafka, exposing a C DL with name-association and an APL producer/consumer API. It supports setting hundreds of standard configs (server, client.id, group.id), producing messages with key+payload, delivery reports, and manual commit for offsets. Demo creates a topic (1 partition), produces two messages, and a consumer in a group reads them in real time; admin tasks use the Kafka CLI. Code is open source on GitHub and a tests branch is under review.
AI-written summary · about
chapters (7)
- 0:00What is Kafka and how does it work?
- 4:40Topics, partitions, and producers
- 8:13Consumers and groups
- 9:51The APL-Kafka interface and API
- 15:21Kafka server administration
- 16:42Demonstration
- 23:12Questions
related talks
50mIntroduction to Apache Kafka
29mGo Big with Apache Kafka
39mKafka as a Platform: The Ecosystem from the Ground Up
41mApache Kafka simply explained
36mApache Kafka Simply Explained
41mKafka as a Platform: the Ecosystem from the Ground Up
45mAdvanced Kafka Features: From Dynamic Consumers to Fault Tolerance
54mKafka as a Platform: the Ecosystem from the Ground Up