Pre-Training

The initial phase of training a large model on a broad, general dataset before it is adapted to specific tasks. Pre-training is computationally expensive and typically done by AI labs with significant resources. The resulting foundation model is then adapted for downstream applications through fine-tuning or prompting.