AlexNet

Neural networks had been dismissed for decades. The 'AI winter' that followed the 1980s hype left the field in disrepute—serious computer scientists worked on other approaches. A small community, including Geoffrey Hinton at the University of Toronto, persisted in developing what they called 'deep learning,' but major conferences routinely rejected their papers. Then in September 2012, a neural network named AlexNet won the ImageNet Large Scale Visual Recognition Challenge by a margin so shocking it changed everything.

ImageNet was the gold standard: a dataset of 1.2 million labeled images across 1,000 categories. Previous winners used hand-engineered feature detectors—edge detectors, color histograms, texture analyzers—carefully designed by computer vision experts. AlexNet, designed by Alex Krizhevsky under Hinton's supervision, used none of these. It learned features directly from raw pixels through eight layers of artificial neurons, trained on two NVIDIA GTX 580 GPUs over five days. Its error rate of 15.3% crushed the runner-up at 26.2%. The gap was unprecedented.

The adjacent possible had assembled perfectly. ImageNet itself had been created only three years earlier—a dataset large enough to train deep networks. NVIDIA's CUDA platform made GPU computing accessible to researchers. Rectified Linear Units (ReLU) solved the vanishing gradient problem that had plagued earlier networks. Dropout regularization prevented overfitting. Each component had existed separately; AlexNet demonstrated they could combine to solve problems previously considered intractable.

The cascade was immediate. Google hired Hinton and his students. Facebook hired Yann LeCun. Every major tech company rushed to build deep learning teams. Within three years, neural networks had surpassed human performance on ImageNet. Within a decade, they had transformed image recognition, speech recognition, language translation, game-playing, and protein structure prediction. The techniques AlexNet popularized—deep convolutional networks, GPU training, dropout—became standard practice.

AlexNet represented a punctuated equilibrium in AI: a sudden shift from one paradigm to another. The field had spent decades on symbolic AI, expert systems, and hand-crafted features. A single result demonstrated that learning from data at scale could outperform human engineering. The researchers who had persisted through the AI winter found themselves leading the next revolution.