Combatting illegal fishing with Machine Learning and Azure
summary
Built a cloud-first pipeline ingesting AIS, satellite and radar, projecting raw data into Parquet partitions in Data Lake Storage, orchestrated by durable Azure Functions and optimized C# to run geospatial arteries indexing for proximity detection and ONNX models for scoring. Custom AIS parser reaches ~7 million records/sec; team benchmarks trying to process 1.5 billion messages/month under a £10/month compute target, and validates huge cost drops via memory optimizations (pre-sized lists, Span<T>, streaming) while producing near‑real‑time fishing alerts.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:42What will learn?
- 3:59Protecting the world's oceans
- 4:54The organisational landscape
- 6:07Cloud evolution
- 7:02Commodity computing
- 9:25Cost comparison
- 12:06Serverless characteristics
- 17:31Developing a feature pipeline
- 20:13Data projections
- 21:10Serverless machine learning
- 21:25Serverless inference
- 21:59Experimental method
- 23:47Proximity algorithm
- 26:51Orchestration
- 27:30Durable Functions
- 35:18Performance in NET
- 36:18Techniques for low allocation compute
- 38:19Reference vs Value types
- 41:18Takeaways
- 43:38References
related talks
23mBuilding AI-Driven Real-Time Lakehouse APIs
29mFighting financial crime: How Kinectify uses Azure Cosmos DB, Fabric, and Azure AI to stop money...
01h 01mStreams, lakes and oceans – working with on big data with Azure ML
25mDeep learning in deep nets: Helping fish farmers feed the world
36mVideo analytics at scale: Deep Learning, Computer Vision, ML & Spark
07mAI-Powered Risk Management
11mEnhancing Coastal Resilience with AI
46mAdvanced RAG with Azure AI Search - top questions answered