AI.dev: Open Source GenAI & ML Summit North America 2023
2023-12-12..2023-12-13
San Jose, United States
as the organisers put it
Register sponsor SUBMIT to speak View the Schedule See Who’s Attending how to access live stream Post Event Report Thank you to all who joined us for AI.dev North America 2023! We look forward to…
who it's for
machine learning engineers, MLOps engineers, platform engineers, data engineers, research scientists, site reliability engineers
topics
retrieval-augmented generation (RAG); vector search with Milvus; LlamaIndex retrieval pipelines; fine-tuning task-specific LLMs; dynamic GPU allocation for ML; kubernetes for model serving; apache cassandra vector search; scaling Whisper speech models; triton inference server deployments; gpu-accelerated dataframes with RAPIDS; multi-agent AI frameworks (AutoGen); reducing LLM hallucinations and evaluation
| title | author | length |
|---|---|---|
| Keynote: How LLMs Have Changed the Way Organizations Do AI | Robert Nishihara | 12m |
Describes concrete lessons from deploying LLMs with Ray and growing hardware fleets, including organizations using dozens, hundreds, or thousands of GPUs; lays… | ||
| Keynote: Enabling AI with Cloud Native | Jorge Castro | 10m |
CNCF frames concrete work to run GenAI at scale: hardware slicing for GPUs/TPUs/FPGAs via dynamic resource allocation, improved NUMA alignment via KEP-3545… | ||
| Keynote: State of Open GenAI | Jim Zemlin | 13m |
Research group launches a September study and finds half of organizations already use generative AI, most plan major investment, and security/privacy are the… | ||
| Keynote: Enterprise Value with Generative AI | Gourab De | 10m |
The presenter feeds the slides’ PDF to ChatGPT to produce the abstract, then argues enterprises face three concrete problems: explosive experiment… | ||
| Keynote : The Open Source AI Alliance | Anthony J. Annunziata | 13m |
A new coalition led by IBM with Meta, the Linux Foundation and 50+ partners including research labs and startups launches the AI Alliance to fund open models… | ||
| Keynote: Building AI with Open Source & Hugging Face | Jeff Boudier | 14m |
Hugging Face positions itself as a mission-driven open platform and claims six surprising facts: over a million repositories with ~400,000 public models and… | ||
| Keynote: Data = AI: Leading the Future of Gen AI with Apache Cassandra | Chet Kapoor | 18m |
Claims Apache Cassandra is the backbone for real Gen AI workloads, citing enterprise users like Uber Netflix and FedEx and multiple customers moving to… | ||
| Keynote: Unlocking Corporate Data with Retrieval-Augmented Generation | Jerry Liu | 10m |
A practical roadmap for moving beyond basic RAG: use LlamaIndex to connect drives, PDFs, databases and APIs to an LLM (quickstart in five lines of code) but… | ||
| Keynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA | Ankit Patel | 12m |
NVIDIA claims 500 open-source projects and notes over 100 million RTX PCs as a target install base. The stack highlights concrete tools: run pandas unchanged… | ||
| Keynote: Accelerating AI/ML Development with Open Source Software | Brian E. Granger | 16m |
AWS describes concrete open source work that lets teams run notebooks as production jobs with Jupyter Scheduler using immutable snapshots and pluggable… | ||
| Keynote: Introducing Apache Cassandra® 5! | Patrick McFadin, Scott Andreas, Ekaterina Dimitrova | 10m |
Cassandra 5 beta1 is released as an Apache release and 5.0 GA is imminent while 5.1 will follow with transactional cluster metadata reintroduced; the 3.x… | ||
| Keynote: Scaling Open Source Large Language Models | Charles Srisuwananukorn | 10m |
Together runs an inference stack (exposed as the Together Inference API) and leverages research like FlashAttention while scaling 100+ open models. Startup… | ||
| Keynote: Teaching LLMs New Knowledge | Sharon Zhou | 05m |
Claims an all-in-one stack that adapts any model with a few lines of code and removes costly data collection/transformation, while adding hundreds of compute… | ||
| Keynote: What's New in AI | Eve Phillips | 12m |
Overview of concrete advances: Transformers (2017) and attention enable long-context models with tens of billions of parameters trained on terabyte-scale… | ||
| Keynote: Next-Gen AI Applications via Multi-Agent Conversation | Victor Dibia | 10m |
AutoGen is an open source multi-agent conversation framework that composes LLM-based agents to plan, act with tools, and communicate via a shared chat history… | ||
| Executorch: Enabling and Accelerating PyTorch Models on Edge Devices | Chen Lai | 33m |
ExecuTorch is a PyTorch 2–native stack that captures models with PyTorch 2 export IR, applies quantization, delegation and memory-planning passes, and… | ||
| Lightning Talk: GPU.x - Dynamic GPU Fraction and Sharing on ML Mainstream Framework | Tiejun Chen | 08m |
Claims a working PoC called GPU.x that injects a transparent Python hooker into mainstream ML frameworks to slice an 8 GB GPU into live fractions (demo shows… | ||
| Crafting Superior ML Pipelines with SapientML | Kosaku Kimura | 17m |
SapientML generates runnable ML pipelines and source code from tabular data and a spec, producing notebooks that include EDA, preprocessing, sampling, and… | ||
| Enter the Brave New World of GenAI with Vector Search | Mary Grygleski | 33m |
Overview of how vector search plugs into GenAI workflows with concrete tooling and a short live demo. Shows Astra (DataStax) managed Cassandra offering with a… | ||
| The Evolution of ONNX: Driving Interoperability in AI... | Sai Kishan Pampana, Ramakrishnan Sivakumar | 30m |
ONNX becomes a practical bridge: TurnkeyML converts PyTorch (example Python) to ONNX, auto-recognizes model inputs, optimizes and benchmarks across runtimes… | ||
| Retrieval Augmentation and Semantic Search at Scale | Ash Vardanian | 32m |
Shows concrete systems and numbers: open‑source USearch and UForm libraries implemented in low‑level code using AVX512 and custom JIT metrics, with benchmarks… | ||
| Lightning Talk: FastTrackML: Experiment Tracking at Hyper Scale | Dave Gantenbein | 07m |
FastTrackML is an open source experiment tracker that replaces MLFlow’s backend to support tens of thousands of jobs in parallel and claims up to 100x faster… | ||
| Lightning Talk: Grounding LLM to Avoid Hallucinations Using Vector S... | Venkata Karthik Penikalapati | 07m |
Claims that grounding LLMs with embeddings plus vector search eliminates hallucinations and provides auditable citations for enterprise use cases such as CRM… | ||
| Human-Centered AI: What I've Learned From Building Two LLM-Powered Products Thi... | Margaret Jennings | 25m |
Built two LLM products this year that prioritize rapid value: start with summarization and retrieval for quick wins, collect interaction signals (the chat UI… | ||
| The Future is Fine-Tuned: Deploying Task-specific LLMs | Devvret Rishi | 39m |
Claim: smaller task-specific LLMs are cheaper and production-ready. Workflow uses Ludwig for declarative fine-tuning and Lorax for dynamic adapter serving with… | ||
| Lunch & Learn VC Panel: Fundraising for Your GenAI Startup in the Current Environment | Avi Bharadwaj, Erica Brescia, Jocelyn Goldfein, Shomik Ghosh | 42m |
Venture interest concentrated in foundation models so far with 60% of AI VC dollars going there, but funds shift toward application and tooling companies that… | ||
| Milvus Vector Database: Unlocking the Future of Open Source Vector Databases | Frank Liu | 30m |
Milvus is an Apache-2 open source vector database built for billion-scale, claiming real-time reads and writes with a distributed microservice architecture… | ||
| Unveil the Magic Without Hoodini: Transform Your Machine Learning Pipelines with Apa... | Nadine Farah | 25m |
Apache Hudi provides upserts and record-level updates plus a timeline and metadata table so pipelines only process deltas instead of full rewrites; example… | ||
| Workshop: GenOps: Building a MLOps Platform to Support GenAI Workloads with Open... | Farshad Ghodsian | 01h 02m |
Shows an end-to-end GenOps platform deployed on GKE using Kubeflow on GKE, MLflow, Spark on Kubernetes, KServe for model serving, Keycloak auth, Istio ingress… | ||
| NeMo: Open Source Toolkit for Conversational AI | Elena Rastorgueva | 23m |
NeMo is an open source conversational AI toolkit (released under Apache 2.0) that ships as pip or Docker packages and pretrained .nemo archives on NGC and… | ||
| Load Management for AI Models - Managing OpenAI Rate Limits with Request Prioritization | Harjot Gill | 31m |
Generates a client-side solution called Aperture that implements a distributed token-bucket (sharded in-cluster) and a weighted fair-queuing scheduler to honor… | ||
| A New Stack for the Gen AI Era | Alan Ho | 39m |
Claims Gemini is natively multimodal and is rumored to have 5× the parameters and 20× the tokens of GPT‑4, producing SOTA benchmarks; the recommended stack… | ||
| Tricks Learned from Scaling WhisperSpeech Models to 80k+ Hours of Speech | Jakub Piotr Cłapa | 35m |
WhisperSpeech is a MIT‑licensed open source TTS trained on tens of thousands of hours using Whisper for semantic tokens, Meta Encodec for compressed audio and… | ||
| Breakdown New Data Silos in Generative AI | Junping Du | 37m |
Open-source Gravitino exposes a centralized metadata lake and a federated query engine that pushes operators into Snowflake and Postgres so cross-cloud joins… | ||
| Consume and Serve Using Caikit and ModelMesh | Rafael Vasquez, Christian Kadner | 31m |
Caikit provides a runtime-and-module toolkit with ready data models and a GitHub template that wraps Hugging Face pipelines into simple REST/gRPC endpoints and… | ||
| Insights from The State of AI Report 2023 | Corina Gurau | 36m |
Annual report distills concrete shifts: proprietary LLMs leap ahead while open models scale usage, 322 million downloads for Llama 2 and 100,000-token context… | ||
| Beyond Transformers - Intro to RWKV Architecture & The World To... | Eugene Cheah, Harrison Vanderbyl | 34m |
Claims a linear-transformer design that cuts inference energy and latency by 10–100x lower inference cost, replaces KV cache with a decayed-sum/matrix state to… | ||
| When AI Became ML, and then AI Again: Tracing the Evolution of Terms and Technologies | Marcus Eagan | 32m |
Tracks how the label shifts: AI originates in 1955, machine learning surfaces by 1959 and gets framed as the pragmatic subset through the booms and winters… | ||
| The Copyright Office is Wrong About AI-Assisted Works | Van Lindberg | 38m |
Claim: the US Copyright Office rejects copyright in Zarya of the Dawn images generated with MidJourney, arguing the outputs are machine-generated and not… | ||
| How Use of Generative AI Affects Both Open Source Developers and their Downstrea... | Roman Shaposhnik | 30m |
Open Source foundations (ASF, LF, Eclipse) publish a shared generative tooling guidance v1.0 that trusts developers but recommends disclosure, because code and… | ||
| Democratization of ML Pipelines and Bringing ML Workflows to Heterogeneous... | Yihong Wang, Tommy Li | 36m |
Kubeflow Pipelines v2 introduces an Intermediate Representation (IR) that is platform‑agnostic and already used by Kubeflow Pipelines, Google Vertex AI and… | ||
| Panel Discussion: The Impact of AI Regulation on Open Innovation in AI | Ben Brooks, Amanda Brock, Peter Cihon, Imo Udom | 01h 08m |
Panels argues that open models enable inspection, fine-tuning and cheaper downstream apps—Stability AI cites Stable Diffusion (~80% of AI images) and released… | ||
| Building a Custom AI Chatbot with Spring Boot, React, and LangChain4j | Marcus Hellberg | 34m |
Builds a full stack RAG chatbot using Spring Boot backend, React + Hilla frontend, LangChain4j and OpenAI GPT-4, with a 500‑token windowed chat memory and a… | ||
| Beyond Parquet and ORC: Upgrading Data Infrastructure for Multi-modal AI with Lance Col... | Chang She | 28m |
Claims a new open columnar format and table layer called Lance that makes AI workloads like vector search, large multimodal training, and EDA far faster and… | ||
| Building Ethical AI: The Power of Open Source and Education | Ofer Hermoni | 33m |
Linux Foundation AI & Data runs 58 open‑source projects (≈22.6M LOC, ~30,000 active contributors) and now spins up a Generative AI Commons three months ago… | ||
| Responsibility in Depth: Layering Licensing, Regulation, and More | Luis Villa | 33m |
Argues that copyright licenses alone fail for open ML, using the RAIL license adoption by Stability — recent press shows models used for non‑consensual porn —… | ||
| Safeguarding Generative AI Applications: Navigating Security Challenges and Best Pract... | Neta Haiby | 27m |
ChatGPT hits 100 million users in two months (versus 16 years for smartphones) and Gartner predicts 80% of enterprises using generative AI by 2026 up from 5%… | ||
| The Power of RAG at Scale: A Year of Explosive Growth and Observab... | Ricardo Aravena, Joshua Reini | 22m |
Claims that Retrieval-Augmented Generation reduces hallucinations but still fails when retrieval returns mixed records are illustrated with a concrete error… | ||
| OpenFL: Building Better AI Models with Private Data | Niroop Ammbashankar | 39m |
OpenFL is an open source federated learning framework that Intel and partners release to run multi‑site training without moving patient data, using gRPC, NumPy… | ||
| Algorithmic Challenges. An Educational Framework for AI & Ethics Literacy | Aldo Pisano | 31m |
An educational framework trains 12 classes in southern Italy over eight-hour modules using civics time, questionnaires and classroom exercises that pit… | ||
| A Deep Dive Into Retrieval-Augmented Generation with LlamaIndex | Laurie Voss | 25m |
LlamaIndex is a Python and TypeScript framework that loads any data source, builds embeddings, and connects to LLMs; it supports 25+ models and 30+ vector… | ||
| Chatbot Arena: An Open Crowdsourced Platform for Human Feedback on LLMs | Wei-Lin Chiang | 27m |
A crowdsourced web platform runs pairwise anonymous battles between chatbots, records over 100K+ votes and millions of served requests, and converts pairwise… | ||
| AutoGen: A Multi-Agent AI Framework | Victor Dibia, Chi Wang | 33m |
AutoGen coordinates multiple specialized agents (LMs, tools or humans) through conversation programming so a planner, coder and reviewer can iteratively write… | ||
| Panel Discussion: Why a Universal Definition of 'Open Source AI' is Essential for Humanity | Roman Shaposhnik, Tanya Dadasheva, Nithya A. Ruff, Sal Kimmich | 29m |
Panel argues for an actionable, cross-industry standard: OSI commits to an OSI definition October 2024 and a license checklist, foundations (Apache, LF) push… | ||
| How to Adopt Emerging Federated Learning Technologies | Myungjin Lee | 30m |
Flame claims to let teams swap and extend federated learning topologies without rewriting ML code by using a Tag topology graph of roles and channels that the… | ||
| Enable Generative AI Everywhere with Ubiquitous Hardware and Open Software | Guobing Chen | 17m |
Implemented low‑precision (bfloat16/INT8/INT4), Flash/Efficient Attention, optimized KV‑cache access, RoPE kernel fusion, tensor parallelism and AMX/EMX CPU… | ||
| Securing LLMs in Kubernetes: Best Practices | Meenakshi Kaushik, Jayanth Srinivasa | 30m |
Deploys an open LLM (uploader lists Mistral) on Kubernetes and front-ends it with NGINX + Istio-style gateways, using an Envoy Lua filter that forwards every… | ||
| End-to-End MLOps with MLflow and Kubeflow | Nick Chase | 30m |
Uses MLflow to log Iris dataset runs with a logistic regression, logging hyperparameter arrays and an accuracy metric, creating an MLflow tracking UI instance… | ||
| Empowering Open-Source Generative AI by Integrating the Wikidat... | Jonathan Fraine, Lydia Pintscher | 31m |
Practical recipe for plugging the Wikidata knowledge graph into generative models: use a Hugging Face NER/intent step to map user tokens to Q‑items, run a… | ||
| Simplify End-To-End MLOps with PostgresML | Montana Low | 32m |
Claims that running ML inside Postgres with PostgresML is far faster and simpler than Python microservices, citing 10x faster end-to-end for embedding+RAG vs… | ||
| The Case for Open Source in Autonomous AI Systems in the Edge | Tina Tsou, Sujata Tibrewala | 33m |
Argues that running autonomous AI at the edge reduces latency and privacy risk and can enable lightweight on-device training instead of relying on year-old… | ||
| The Future of Open Source AI & The Generative AI Commons | Matt White | 34m |
Open source generative AI is moving toward domain-specific LLMs, RAG pipelines using Milvus and Cassandra, ensemble-of-experts architectures and automated… | ||
| Open Source DataOps and MLOps Strategies | Lisa Cao | 23m |
Outlines a practical path for open source DataOps and MLOps that centers on measurable wins: prioritize four domains—accuracy and data quality, visibility and… | ||
| Scaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App... | Jack Min Ong | 32m |
Kubernetes uses the NVIDIA GPU Operator plus the device plugin to label nodes, enable a Mig manager, and expose GPU resources; MIG physically shards cards into… | ||
| Streamlining RAG Applications with Pinecone's OSS Framework: Canopy | Audrey Lorberfeld | 27m |
Canopy is Pinecone’s open source RAG framework that automates chunking, embedding, indexing and prompting so a developer only needs a Pinecone API key and an… | ||
| Leveraging SRE and Observability Techniques for the Wild World of Building on LLMs | Christine Yen | 33m |
Honeycomb builds a query assistant on top of OpenAI APIs with about six weeks development and eight weeks iteration, instruments every prompt and response, and… | ||
| Reducing Hallucinations and Evaluating LLMs for Production | Divyansh Chaurasia | 26m |
Shows an actionable pipeline: map docs into vector DBs, build a RAG or LangChain stack, create a manual evaluation set of hundreds of prompts, then automate… | ||
| Lightning Talk: Turning the Internet Into 3D: 3D Content with NeRF and Gaussian Splat... | Barkley Dai | 18m |
Luma AI turns phone photos into interactive 3D captures by producing both NeRF reconstructions and fast Gaussian Splatting point‑cloud views so scenes load in… | ||
| Using Generative AI to Develop Open Source Code - Practical Guidance for Managing Lega... | Joanna Lee | 31m |
Practical legal guidance on using generative AI for open source code: copyright requires human authorship so edits matter, training data can be reproduced and… | ||
| Lightning Talk: AI Penetration Testing | Blake Turrentine | 07m |
Covers two red team approaches: ethical hacking of others models and adversarial red teaming. Demonstrates file fuzzing via bit-flipping file fuzzing against… | ||
| Workshop: Empowering Collaboration: AI Developer Experience - Your Bridge from Mod... | Cedric Clyburn | 58m |
Shows an end‑to‑end GenAI pipeline that fine‑tunes Stable Diffusion 1.5 with DreamBooth on images from an S3 bucket, exports to ONNX, runs a… | ||
| Workshop: Prompt Engineering with Watsonx | Rafael Vasquez | 37m |
Hands-on walkthrough of Watsonx Prompt Lab that shows live edits beating guesswork: swap models (flan_ul2, Llama 2, StarCoder), raise max tokens to avoid… | ||
| Workshop: Log Processing with Apache NiFi and Apache Kafka | Sophia Izokun | 46m |
Hands-on workshop shows NiFi ingesting the NASA Kennedy Space Center Logs (≈3.5 million requests), splitting the ASCII file into one-log-per-line with… | ||
| What Is It Like to Be an LLM? | Matt Butcher | 29m |
Claims that modern LLMs produce plausible text by pattern‑matching across massive corpora, not by lived experience, so they have no phenomenology and no… | ||
| Workshop: LLMs Fine Tuning and Inferencing Using ON... | Abhishek Jindal, Sunghoon Choi, Kshama Pawar | 01h 02m |
Fine-tuning and inference use ONNX Runtime with Azure Container for PyTorch, AzureML, DeepSpeed stage 2 and LoRA on Mistral 7B v0.1. Training on 8x V100 32GB… | ||
| Workshop: Define "Open AI" | Mer Joyce, Ruth Suehle | 20m |
Hands-on workshop that builds a concrete license checklist mapping the four software freedoms to the components of AI systems, breaking work into small groups… | ||
| Keynote: Learnings from Studying 800 Generative AI Open Source Repos | Chip Huyen | 12m |
Scans GitHub down from ~26,000 LM hits and ~100,000 GPT hits to a curated set of 800 repos (>=5–6 stars), categorizes them into stack layers and finds most… | ||
| nothing matches | ||