Back to glossaryExternal reference
AI GLOSSARY
Instrumental Convergence
Safety, Alignment & Ethics
The observation, formalized by Nick Bostrom, that a wide range of AI systems with very different ultimate goals would likely converge on certain intermediate goals, such as self-preservation, resource acquisition, and resistance to goal modification, because these are useful for achieving almost any objective. Instrumental convergence is a key concept in AI safety: it suggests that even an AI with seemingly benign goals could develop concerning behaviors as a side effect of pursuing them effectively.
See also: control problem, corrigibility, AI alignment.