Inference Cost

The computational cost of running a trained AI model to generate outputs. Every user interaction triggers inference, and at scale these costs can become significant, making inference cost a key factor in decisions about which models to deploy, at what context length, and under what pricing structure.
See also: inference, FLOPS, hosted model.