Keynote: LLM Research and Development at National Institute of Informatics
summary
LM-JP builds fully open models with training code, checkpoints and a searchable training corpus; the first release is LJP-13B (GPT-2 style, 13B params) trained on ~300B tokens with ~50% Japanese and custom 50k vocab, ~30k downloads so far. Tuning uses ~200k instruction samples (JASA, Dory, OpenAssistant) plus ~1,000 safety items; architecture shifts to LLaMA with 100k vocab in v2. A 175B experiment ran and a 172B planned release targets public distribution; >1,300 contributors across 160 organizations participate.
AI-written summary · about
chapters (9)
- 0:00LLM research at NII
- 3:30How LLMs work
- 8:50LLM history and evolution
- 15:20LLM challenges
- 18:20LM JP launch
- 21:10LM JP development
- 24:20LM JP safety
- 27:00Future plans and collaboration
- 28:20NII mission
related talks
50mLook Inside a Large Language Model to Become a Better Java Developer
37mExpanding the Capabilities of LLMs Through Community Collaboration
27mProductionizing Open Source LLMs
01h 02mA hacker's guide to open source LLMs
05mKeynote: Teaching LLMs New Knowledge
15mKeynote: Common Corpus: Opening Data for Building Open Source LLMs
24mHarnessing the Power of LLMs in Advanced Analytics
33mOpening Keynote: LLMs, between fantasy and reality - the poin...