Facebook AI head foresees field hitting a brick wall

In an interview with Wired, Facebook VP of AI Jerome Pesenti talked about the current state of the technology as well as some of the new problems it generates like deep fakes. Pesenti also noted that the current AI and deep learning techniques have a lot of limitations, and that at some point we're going to hit the wall as the current rate of progress is not sustainable due to scaling issues:

WK: OpenAI recently noted that the compute power required for advanced AI is doubling every 3 and a half months. Are you worried about this?

Jerome Pesenti
JP: That’s a really good question. When you scale deep learning, it tends to behave better and to be able to solve a broader task in a better way. So, there's an advantage to scaling. But clearly the rate of progress is not sustainable. If you look at top experiments, each year the cost it going up 10-fold. Right now, an experiment might be in seven figures, but it’s not going to go to nine or ten figures, it’s not possible, nobody can afford that.

It means that at some point we're going to hit the wall. In many ways we already have. Not every area has reached the limit of scaling, but in most places, we're getting to a point where we really need to think in terms of optimization, in terms of cost benefit, and we really need to look at how we get most out of the compute we have. This is the world we are going into.

You can read more at Wired.