Back to glossary
AI GLOSSARY
Top-K Sampling
Large Language Model (LLM) Terms
A decoding strategy that restricts token selection to only the K most probable next tokens, sampling randomly from that shortlist. It prevents the model from choosing very unlikely tokens while still introducing variety, and the value of K controls the tradeoff between diversity and coherence.
See also: top-p sampling, sampling.