Disparate Impact

The phenomenon where an AI system produces outcomes that disproportionately disadvantage a particular group, even if the system was not explicitly designed to discriminate. Disparate impact can arise from biased training data, proxy variables that correlate with protected characteristics, or optimization objectives that fail to account for equity. It has legal significance in many jurisdictions: in the US, disparate impact is a recognized theory of discrimination under civil rights law.
See also: demographic parity, algorithmic bias, bias (data).