Value Alignment

The specific challenge of ensuring that an AI system's values, the things it implicitly optimizes for through its behavior, match the values of the humans it is meant to serve. Value alignment is broader than just following instructions, it requires the system to have internalized a sufficiently rich and accurate model of human values to behave appropriately even in novel situations that its designers did not anticipate.