How to train GPT-3 with your own dataset
summary
GPT-3 (175 billion parameters) gets fine-tuned via the OpenAI API using a JSONL dataset; a 5,000-document corpus yields prompts but fine-tuned with a few hundred examples transfers the base model into a domain and finishes training in about four to five hours. A working strategy bot for Ashoka generates multi‑paragraph strategy papers that are useful but still prone to hallucination, so the deployed workflow pairs model output with human review; testing done in the GPT playground and via the completion API.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:17What is GPT-3?
- 6:40How to use GPT-3-One shot learning
- 7:14How to use GPT-3-Zero shot learning
- 8:16Project "Strategy Bot" with Ashoka
- 9:44Finetuning GPT-3
- 12:11Data preparation
- 13:18Example prompt
- 17:44Additional Material
related talks
39mBuilding Your Own Bot with GPT: Harnessing the Power of AI...
51mOpenAI and the AI that may overtake humans
28mGPT or how I learned to stop worrying and love the generative AI
25mThe Story of OpenAI GPT3 - Improve Observability with Tracing
36mHow NOT to Train Your Hack Bot: Dos and Don'ts of Building Offensive GPTs
39mCollect Data to Fine-Tune Your Large Language Model
43mState of GPT
45mBringing the Power of AI to Your Application