Activation Function

A mathematical function applied to the output of a neuron in a neural network, determining whether and how strongly that neuron fires. In practice, a neural network passes data through repeated cycles of linear transformation followed by an activation function, and it is this alternation that gives the network its ability to learn complex, non-linear patterns. Without activation functions, stacking layers would be mathematically pointless, as any number of linear transformations collapse into a single one.