🧱 Basics
Training
Pre-training, pre-training
The process in which a model learns from data by adjusting its parameters step by step. For language models: pre-training on huge text corpora, then refinement.
💭 Imagine …
Years of reading a gigantic library: expensive, slow and done once. Afterwards, the knowledge is only applied.
🔎 Example
Pre-training a large model runs for weeks to months on thousands of GPUs.
🎯 Why it matters
What wasn't in the training data, the model can't know. Whether your inputs are used for future training depends on the provider; business tools usually exclude it by contract.
🎓 Prefer step by step?
The AI Literacy course: 6 short modules with quizzes, a solid foundation in about 60 minutes.