NVIDIA chief scientist and senior vice president of research Bill Dally argues the CPU scaling predicted by Moore's Law has hit a brick wall, and that a leap into parallel processing is necessary to keep advancing the computing industry at a rapid pace.
The good news is that there is a way out of this crisis. Parallel computing can resurrect Moore's Law and provide a platform for future economic growth and commercial innovation. The challenge is for the computing industry to drop practices that have been in use for decades and adapt to this new platform.
Going forward, the critical need is to build energy-efficient parallel computers, sometimes called throughput computers, in which many processing cores, each optimized for efficiency, not serial speed, work together on the solution of a problem. A fundamental advantage of parallel computers is that they efficiently turn more transistors into more performance. Doubling the number of processors causes many programs to go twice as fast. In contrast, doubling the number of transistors in a serial CPU results in a very modest increase in performance--at a tremendous expense in energy.