General-purpose computing on GPU

Graphics chips were supposed to draw dragons, not train language models. Yet the same hardware built to shade pixels for games turned out to be unusually good at another job: doing the same arithmetic operation thousands of times at once. That accidental fit is what made general-purpose computing on GPU, usually shortened to GPGPU, feel inevitable once the right software arrived.

The hardware substrate came first. During the late 1990s and early 2000s, the `gpu` evolved into a massively parallel processor because real-time 3D graphics demanded it. Rendering a frame meant applying similar mathematical operations across huge numbers of vertices, fragments, and textures. Games paid for the silicon. Semiconductor scaling and graphics competition delivered chips with hundreds of arithmetic units before most mainstream software knew what to do with them. Scientists and graphics researchers saw the mismatch immediately: here was cheap parallel hardware sitting on millions of desks, but it could only be programmed indirectly through graphics APIs.

Stanford's BrookGPU project showed the first workable bridge. In 2004, Ian Buck and collaborators presented Brook as a C-like language for stream computing on programmable graphics hardware, reporting workloads such as FFT and ray tracing running up to seven times faster than CPU versions. That result mattered because it translated a graphics trick into a computing method. Researchers no longer had to pretend matrix multiplication was an image-processing problem quite so explicitly. Still, Brook was a research tool living inside the constraints of shader languages, texture memory, and graphics-driver workarounds. The idea was alive, but the habitat was still hostile.

Commercial emergence came when `nvidia` decided that selling GPUs to scientists and engineers might be as important as selling them to gamers. CUDA appeared in 2006 and reached a public 1.0 release on June 26, 2007. A few days earlier, NVIDIA had announced the Tesla line of GPU computing products, including boards and deskside systems pitched as personal supercomputers. That pairing mattered more than any single chip. CUDA gave programmers a direct, C-like way to write kernels for the GPU; Tesla gave institutions a product line they could buy for finance, molecular dynamics, oil and gas, and computer vision. GPGPU stopped being a lab stunt and became a market.

`amd` moved along a parallel path almost immediately. Its November 8, 2007 FireStream 9170 announcement promised up to 500 gigaflops from a board built for scientific and engineering workloads, and the accompanying SDK exposed both low-level access and Brook+-style higher-level tools derived from the Stanford work. That is why GPGPU is a good example of `convergent-evolution`: Stanford researchers, NVIDIA product teams, and AMD's post-ATI stream-computing effort were all circling the same discovery from different directions. Once programmable shaders and cheap parallel silicon existed, someone was going to try to free them from graphics.

What made the shift durable was repurposing. GPUs had not been designed for numerical simulation or neural-network training. They had been selected for texture filtering, rasterization, and game performance. But a chip optimized for doing many similar floating-point operations at once could be reassigned to dense linear algebra, particle simulations, and eventually deep learning. GPGPU was not a brand-new machine from first principles. It was old graphics hardware moved into a richer ecological niche.

Open standards widened that niche. Apple proposed OpenCL in 2008, and the Khronos Group ratified OpenCL 1.0 in Singapore on December 9, 2008 with backing from Apple, AMD, NVIDIA, Intel, and others. OpenCL mattered because it told developers that heterogeneous computing was bigger than one vendor's toolchain. Yet `path-dependence` had already started to bite. CUDA had an earlier lead, a growing code base, teaching materials, and hardware tuned around its users. OpenCL expanded the field, but it did not erase the advantage of the first ecosystem that felt usable.

Then `niche-construction` took over. Once programmers had workable languages, libraries, and boards, they began building software that assumed GPU acceleration existed. Scientific codes were ported. Finance and imaging workloads moved over. Then machine learning hit the same substrate with much greater force. `alexnet` in 2012 used a GPU implementation of convolutional networks to produce the result that reset computer vision. `generative-adversarial-network` research and `transformer-machine-learning` later depended on the same basic bargain: if your model can be expressed as huge batches of linear algebra, a GPU cluster can train it at a scale CPUs would price out or slow down. At that point GPGPU was behaving like a `keystone-species` invention inside computing: remove it and large parts of modern AI shrink with it.

General-purpose computing on GPU therefore marks a hinge rather than a gadget. It was the moment graphics hardware ceased to be a special-purpose peripheral and became shared computational infrastructure. Gaming demand financed the silicon, academic work exposed the possibility, NVIDIA and AMD turned that possibility into products, and software ecosystems locked in the trajectory. After that, AI did not need a wholly new machine. It inherited one that had already learned how to run in parallel.