GPT-4 is a Transformer-based model introduced by OpenAI in March 2023. The original multimodal model accepts text and image inputs and produces text outputs.
Main changes
- Adds image input to text input
- Improves performance on many academic benchmarks compared with GPT-3 and GPT-3.5
- Produces more detailed answers on complex tasks
Model size and training data
| Claim | Status |
|---|---|
| 1.7 trillion parameters | Unconfirmed estimate |
| 45 terabytes of training data | Unconfirmed claim |
| Exact parameter count and training-data size | Omitted from the public technical report |
The public report describes next-token pre-training and RLHF fine-tuning. It withholds the detailed architecture and model size. GPT-4 technical report
Treat the size estimates as uncertain. Compare published capabilities directly when studying GPT-3 and GPT-4.
See Generative Pre-trained Transformer (GPT) for the model family and ChatGPT for the dialogue system.