Back to glossary

AI GLOSSARY

Top-K Sampling

Large Language Model (LLM) Terms

A decoding strategy that restricts token selection to only the K most probable next tokens, sampling randomly from that shortlist. It prevents the model from choosing very unlikely tokens while still introducing variety, and the value of K controls the tradeoff between diversity and coherence.
See also: top-p sampling, sampling.