Tag Archives: Moore’s Law As It Applies to Artificial Intelligence

Close-up of a glowing microchip on a dark blue circuit board, highlighting intricate electronic components.

Moore’s Law As It Applies to Artificial Intelligence – Part 2/2

Artificial Intelligence, Categories, Technology, , ,

As previously mentioned in part 1 of this blog post, Moore’s law is not a physical law of science. Rather it may be considered a trend or a general rule. This begs the following question. How Long Will Moore’s Law Hold?

There are numerous estimates regarding how long Moore’s law will hold. Since it is not a physical law, its applicability is routinely questioned. For approximately the last half century, each estimate, at various points in time, has predicted that Moore’s law would hold for another decade. This has been occurring for almost five decades.

In 2005 Gordon Moore stated in an interview that Moore’s law “can’t continue forever. The nature of exponentials is that you push them out and eventually disaster happens.” Moore noted that transistors eventually would reach the limits of miniaturization at atomic levels. “In terms of size [of transistors] you can see that we’re approaching the size of atoms, which is a fundamental barrier, but it’ll be two or three generations before we get that far—but that’s as far out as we’ve ever been able to see. We have another 10 to 20 years before we reach a fundamental limit.”

However, new technologies are emerging to use molecules individually positioned, replacing transistors altogether. This means computer “switches” will not be transistors but molecules. The position of the molecules will be the new switches. This technology is predicted to emerge by 2020 (Baptiste Waldner, Nanocomputers and Swarm Intelligence, 2008).

Some see Moore’s law extending far into the future. Lawrence Krauss and Glenn D. Starkman predicted an ultimate limit of around six hundred years (Lawrence M. Krauss, Glenn D. Starkman, “Universal Limits of Computation,” arXiv:astro-ph/0404510, May 10, 2004).

I worked in the semiconductor industry for more than thirty years, during which time Moore’s law always appeared as if it would reach an impenetrable barrier. This, however, did not happen. New technologies constantly seemed to provide a stay of execution. We know that at some point the trend may change, but no one really has made a definitive case as to when this trend will end. The difficulty in predicting the end has to do with how one interprets Moore’s law. If one takes Moore’s original interpretation, which defined the trend in terms of the number of transistors that could be put on an integrated circuit, the end point may be somewhere around 2018 to 2020. Defining it in terms of “data density of an integrated circuit,” however, as we did regarding AI, removes the constraint of transistors and opens up a new array of technologies, including molecular positioning.

Will Moore’s law hold for another decade or another six hundred years? No one really knows the answer. Most people believe that eventually the trend will end, but when and why remain unanswered questions. If it does end, and Moore’s law no longer applies, another question emerges.

What Will Replace Moore’s Law?

Ray Kurzweil views Moore’s law in much the same way we defined it, not tied to specific technologies but rather as a “paradigm to forecast accelerating price-performance ratios.” From Kurzweil’s viewpoint:

 Moore’s law of Integrated Circuits was not the first, but the fifth paradigm to forecast accelerating price-performance ratios. Computing devices have been consistently multiplying in power (per unit of time) from the mechanical calculating devices used in the 1890 U.S. Census, to [Newman’s] relay-based “[Heath] Robinson” machine that cracked the Lorenz cipher, to the CBS vacuum tube computer that predicted the election of Eisenhower, to the transistor-based machines used in the first space launches, to the integrated circuit-based personal computer. (Raymond Kurzweil, “The Law of Accelerating Returns,” www.KurzweilAI.net)

In the wider sense, Moore’s law is not about transistors or specific technologies. In my opinion it is a paradigm related to humankind’s creativity. The new computers following Moore’s law may be based on some new type of technology (e.g., optical computers, quantum computers, DNA computing) that bears little to no resemblance to current integrated-circuit technology. It appears that what Moore really uncovered was humankind’s ability to cost-effectively accelerate technology performance.

Source: The Artificial Intelligence Revolution (2014), Louis A. Del Monte

Close-up of a glowing microchip on a dark blue circuit board, highlighting intricate electronic components.

Moore’s Law As It Applies to Artificial Intelligence – Part 1/2

Artificial Intelligence, Categories, Technology, , , ,

Intel cofounder Gordon E. Moore was the first to note a peculiar trend, namely that the number of components in integrated circuits had doubled every year from the 1958 invention of the integrated circuit until 1965. In Moore’s own words:

The complexity for minimum component costs has increased at a rate of roughly a factor of two per year.…Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000. I believe that such a large circuit can be built on a single wafer. (Gordon E. Moore, “Cramming More Components onto Integrated Circuits,” Electronics magazine, 1965)

 In 1970 Caltech professor, VLSI pioneer, and entrepreneur Carver Mead coined the term “Moore’s law,” referring to a statement made by Gordon E. Moore, and the phrase caught on within the scientific community.

In 1975 Moore revised his prediction regarding the number of components in integrated circuits doubling every year to doubling every two years. Intel executive David House noted that Moore’s latest prediction would cause computer performance to double every eighteen months, due to the combination of not only more transistors but also the transistors themselves becoming faster.

From the above discussion, it is obvious that Moore’s law has been stated a number of ways and has changed over time. In the strict sense, it is not a physical law but more of an observation and guideline for planning. In fact many semiconductor companies use Moore’s law to plan their long-term product offerings. There is a deeply held belief in the semiconductor industry that adhering to Moore’s law is required to remain competitive. In this sense it has become a self-fulfilling prophecy. For our purposes in understanding AI, let us address the following question.

What Is Moore’s law?

As it applies to AI, we will define Moore’s law as follows: The data density of an integrated circuit and the associated computer performance will cost-effectively double every eighteen months. If we consider eighteen months to represent a technology generation, this means every eighteen months we receive double the data density and associated computer performance at approximately the same cost as the previous generation. Most experts, including Moore, expect Moore’s law to hold for at least another two decades, but this is debatable, as I discuss later in the chapter. Below is a graphical depiction (courtesy of Wikimedia Commons) of Moore’s law, illustrating transistor counts for integrated circuits plotted against their dates of introduction (1971–2011).

As previously mentioned, Moore’s law is not a physical law of science. Rather it may be considered a trend or a general rule. This begs the following question. How Long Will Moore’s Law Hold? We will address this and other questions in part 2 of this post.

Source: The Artificial Intelligence Revolution (2014), Louis A. Del Monte