Scaling From Research to Production with Skymind DL4J and ScalNet
summary
Shows how DL4J/ScalNet takes a Keras/TensorFlow prototype into a JVM production pipeline: ingest via DataVec with on-the-fly batch transforms, ND4J off‑heap arrays, and an LSTM temporal IoT example trained and evaluated across Spark with parameter averaging or an Akka-based trainer. Demonstrates Keras H5 import (images work, LSTM sequence-to-dense has current limitations), transfer learning helpers, parallel training via Spark or Akka, FP16 support and model serialization for serving.
AI-written summary · about
chapters (24)
- 0:00Intro
- 0:39Overview
- 1:49Open Source
- 2:36History
- 7:41Use Cases
- 8:31Datavac
- 10:08Training Data
- 11:20Balancing Data
- 18:25Multi Data Sets
- 19:40Models
- 21:07Model Zoo
- 22:04MultiLayer vs Computation Graph
- 23:29Skele
- 26:28Transfer Learning
- 27:54Learning Helper
- 29:32Evaluation
- 31:22Monitoring
- 32:46Parallelization
- 33:17Parallel Inference
- 34:02MultiCPU
- 34:42Floating Precision
- 35:02Parallel Wrapper
- 35:27Apache Spark
- 36:20Apache Training Master
related talks
44mDeep Learning in Production
51mDeep Learning with DL4J and Apache Spark?
40mScala and Spark in the New Deep Learning Era
02h 37mDeveloping Cloud-Native Java AI applications with DJL and LangChain4J
23mRethinking scalable machine learning with Spark ecosystem
50mDeep Learning and NLP with Spark
45mDeep Learning and NLP with Spark
35mJVM based DeepLearing on IoT data with Apache Spark