How close are we to universal AI?

This has been a recurring theme in science fiction for decades, but thanks to the rapid advances in artificial intelligence in the past few years, the debate has been revived and intensified. More and more media and mainstream voices are warning us of the arrival of general artificial intelligence, claiming that the process will happen much faster than we think. For example, a new documentary, “Can you Trust This Computer? Funded by Elon Musk, the premiere attracted AI experts from academia and industry. The documentary paints a stunning picture of artificial intelligence as “new life forms on Earth” that will surround us “with their tentacles.” A growing number of stories point to a scary aspect of AI: some of the alternatives to reality (fake celebrity face generators and Deepfakes, possibly with video generation and voice synthesis in the near future), Boston Dynamics’ quirky videos (the latest: Robots collaborating to open a door), and reports that Google’s AI has become “highly aggressive.”

However, as an investor who has spent a lot of time in the trenches of AI, I have experienced quite a bit of cognitive dissonance on this topic. I talk to a lot of AI entrepreneurs every day, and the story I learned is quite different: even if you solve a specific problem, hire a professional machine learning engineer, and raise millions of dollars in venture capital, it’s still very hard to build an AI product for the real world. It is clear that even “narrow” AI is far from working when it needs to be executed 100% correctly in the real world, as the recent deaths caused by autopilot have tragically demonstrated.

So what is the reality? Exponential advances in technology made general ai look like it was in the attainable future, but suddenly we found out it wasn’t. Are we approaching an inflection point?

A lot of articles on AI are about building AI applications and startups, and in this article I seem to be swimming upstream in the world of AI research, trying to understand who is doing what and what new things might be invented in the AI research lab. I had the pleasure of attending a great Workshop a few weeks ago, the Canonical Computation in Brains and Machines symposium at NYU, which was particularly enlightening for me and the main source of this article.

There have been reports of an explosion in AI entrepreneurial activity, with $15.2 billion of venture capital flowing into AI startups, according to a 2017 report, but the same is true upstream of AI research.

The total number of AI papers has increased dramatically since 2012, and there are even projects like Arxiv Sanity Preserver, a browser that can access more than 45,000 papers, launched by Andrej Karpathy because “things have gotten really out of control”.

NIPS is a high-level academic conference that began in 1987. It was once a small, little-known event, but by 2017, 8,000 people had attended.

Ai research is becoming increasingly global. In addition to US universities (such as MIT’s CSAIL Lab), some of the most advanced AI research centers are located in Canada (notably Toronto, the University of Toronto and the new Vector Institute, as well as Montreal, including MILA), Europe (London, Paris, Berlin), and Israel. Moreover, there are more and more Chinese scholars. Interestingly, the AI academia is starting to see an increasing number of brilliant young scholars, including some teenagers, who are tech-savvy and forward-thinking, presumably as a result of the democratization of AI tools and education.

Another major trend is that more and more basic AI research has found its way into large Internet companies. Of course, the model of company-sponsored LABS, like Bell LABS, is not new. But that model is a completely different landscape in AI. Alphabet and Google each have Deepmind, a 2014 startup that now has a team of 700 people focused on basic ARTIFICIAL intelligence, Run by Demis Hassabis) and Google Brain (founded in 2011 by Jeff Dean, Greg Corrado, and Andrew Ng, with a greater focus on artificial intelligence applications). Facebook has FAIR LABS, led by Yann LeCun, one of the fathers of deep learning. Microsoft has MSR AI. Uber owns Uber AI LABS, which it acquired from New York startup Geometric Intelligence. Alibaba has Alibaba AI Lab, Baidu Has Baidu Research Institute, and Tencent has Tencent ARTIFICIAL Intelligence Lab.

These industrial LABS have deep resources and can pay millions of dollars to attract top researchers. A recurring theme in conversations with AI researchers is that if startups struggle to attract students with Ph.D.s in machine learning, academia will find it harder to retain them.

Many of these LABS are explicitly or implicitly pursuing AGI.

In addition, AI research, especially in industrial laboratories, has access to two crucial resources: data and computing power.

There is an increasing amount of data available to train AI, and Internet giants like Google and Facebook have a big advantage in developing general-purpose AI solutions. It’s a similar story in China, where huge data pools are being pooled together to train smart Face recognition, with unicorn startups like Megvii (aka Face+ +) and Sensetime being the biggest beneficiaries. In 2017, China launched a program called “Xueliang Project” to centralize surveillance and processing of surveillance cameras (public and private) from more than 50 Chinese cities.

In addition to data, another change that can cause AGI is the huge acceleration in computing power, especially in recent years. This is the result of progress in taking existing hardware and building new high-performance hardware specifically for AI at a rate that exceeds Moore’s Law.

The team that won the ImageNet competition in 2012 used two Gpus to train their network model. It took five to six days at the time and was already considered the fastest training speed. In 2017, Facebook announced that they had been able to train ImageNet using 1 GPU in 256 hours. Just a few months later, a Japanese team from Preferred Networks broke the record, training ImageNet in just 15 minutes using 1024 NVIDIA Tesla P100 Gpus.

But this could be just a warm-up exercise, as the world is now racing to build ever more powerful AI chips and the hardware that surrounds them. In 2017, Google released a second-generation Tensor Processing Unit (TPU) specifically designed to accelerate machine learning tasks. Each TPU can provide 180 teraflops of performance (for reasoning and training of machine learning models). These TpUs can be clustered to produce supercomputers — systems of 1,000 cloud TPUs that can be used by AI researchers willing to share their work publicly.

There are also plenty of active, well-funded hardware start-ups in the startup world, like Cerebras, Graphcore, Wave Computing, Mythic, and Lambda, as well as Chinese startup Horizon Robotics, Cambricon and DeePhi.

Finally, new hardware innovations are emerging around quantum and optical computing. It’s early days from a research standpoint, but Both Google and IBM are making meaningful advances in quantum computing that could take AI to the next level.

Huge increases in computing power have opened the door to training artificial intelligence with ever larger amounts of data. It also enables AI researchers to run experiments faster, accelerate progress, and create new algorithms.

One point OpenAI (Elon Musk’s nonprofit research lab) has been making is that the power of ARTIFICIAL intelligence has surprised us since algorithms were running on relatively ordinary hardware five years ago — who knows what will happen with all the computing power we have now? (See Greg Brockman, CTO of OpenAI blog Towards Artificial General Intelligence with Greg Brockman on TWiML & AI.)

The 2012 ImageNet competition, driven by deep learning, has led to a stunning resurgence in ARTIFICIAL intelligence. This statistical technique, pioneered and refined by several ai researchers, including Geoff Hinton, Yann LeCun, and Yoshua Bengio, involves multi-level processing of progressively improved results (published in Nature in 2015: Deep Learning). It’s an ancient technology, dating back to the 1960s, ’70s and’ 80s, but it suddenly shows its real power when it provides enough data and computing power.

Almost every AI product has been driven by deep learning, from Alexa to the use of AI in radiology to the “Hot Dog or Not” spoof on HBO’s Silicon Valley. Deep learning has proven to be very effective in a variety of speech recognition, image classification, target recognition, and some language problems.

From the point of view of AGI, deep learning inspired imagination, because it can do far more than it was the range of programming, for example, let the machine around ideas grouped images or words such as “New York” and “us”), without explicitly told that there is a link between these images or words (such as “New York in the United States”). Ai researchers themselves don’t exactly understand why deep learning works.

Interestingly, however, while others are beginning to broadly embrace deep learning for applications from consumers to enterprises, the AI research community wonders if it is on the decline. Geoff Hinton himself sent shock waves through the field of AI research at a conference in September 2017 when he questioned back propagation, which is at the heart of his neural network, and suggested starting from scratch. Gary Marcus, in a January 2018 paper, raised ten concerns about deep learning and argued that “deep learning must be complemented by other technologies if we are to reach universal AI”.

Much of the discussion seems to focus on “supervised” learning — samples that need to display a large number of markers to train machines to recognize similar patterns.

The AI research community now seems to agree that more effort and attention needs to be paid to unsupervised learning — training without tagging data — if we are to achieve AGI. There are many variants of unsupervised learning, including autoencoders, deep confidence networks, and GAN.

GAN, or “generative adversarial networks,” is a relatively recent approach that is directly related to unsupervised deep learning, pioneered by Lan Goodfellow in 2014 when he was a PhD student at the University of Montreal. GAN trains on the same data by establishing competition between two neural networks. A network (generator) produces as realistic an output as possible (such as a photo); Another network (discriminator) compares the photos to the training data set and tries to distinguish whether each photo is real or fake; The generator then adjusts its parameters, produces a new image, and keeps looping. GAN has already had its own variations. In 2017, there were several versions of GAN (WGAN, BEGAN, CycleGan, Progressive GAN) in one year. In the final approach, NVIDIA used gans to generate high-resolution photos of fake celebrities’ faces by gradually training them.

Another related area that is growing at a similar pace is reinforcement learning — the AI can separate the good action (which can be rewarded) from the bad by trying to teach itself how to do something over and over again, changing its approach each time until it has mastered the action. Reinforcement learning, another technique that dates back to the 1950s, has long been considered an interesting but not very useful idea.

That all changed at the end of 2013, however, when DeepMind, an independent startup, taught AI to play 22 Atari 2600 games, all of them at a level that surpassed humans. In 2016, AlphaGo, an AI that has undergone intensive learning, beat South Korean go master Lee Sedol. Just a few months ago, in December 2017, AlphaZero, a more general and powerful version of AlphaGo, used the same method to master not only Go, but also chess and checkboard. With no guidance from anyone other than the rules of the game, AlphaZero taught itself how to be a chess master in just four hours. Within 24 hours, AlphaZero was able to beat all of the current most advanced AI programs (Stockfish, Elmo, and the three-day version of AlphaGo) in all three games.

How close is AlphaZero to Universal AI? Demis Hassabis, DeepMind’s chief executive, called AlphaZero’s style “alien” because it won games entirely by counterintuitive actions, such as sacrificing pieces. It’s a daunting experience to watch a computer program hone the most complex human game to a world-class level in just a few hours, approaching some form of intelligence.

One counter-theory that has emerged in the AI world is that AlphaZero’s training process is actually a brute force algorithm: AlphaZero trains by playing against itself with 5,000 first-generation TPus and 64 second-generation TPus. Once the training is complete, it needs to be run on a machine with 4 TPus. In reinforcement learning, AI researchers point out that the AI does not know what it is actually doing (such as playing a game), but is only constrained by the specific constraints it is given (the rules of the game). Check out this blog post: Is AlphaZero really a scientific breakthrough? .

When it comes to AGI, or even machine learning in general, some researchers have high hopes for migrated learning. DeepMind’s Deavy Hasabi, for example, calls transfer learning “the key to general intelligence”. Transfer learning is a machine learning technique in which models trained on one task are repositioned on a second related task. The idea is that with prior knowledge learned from the first task, the AI can perform better, train faster and require less marker data than a new neural network trained from scratch on the second task. Basically, it is hoped that it will help AI become more “generic”, from task to task and domain to domain, especially in cases where marker data is less readily available (see Overview: Transfer Learning-Machine Learning’s Next Frontier).

If YOU want to implement AGI relying on transfer learning, the AI needs to be able to transfer learning across increasingly distant tasks and domains, which will require increased abstractions. According to Hassabis, “The key to transfer learning is to acquire conceptual knowledge that is abstracted from the perceptual details that you learn.” We’re not at that stage yet. Transfer learning has always been a challenging task — it works well when tasks are closely related, but beyond that, things get much more complicated. But this is a key area of artificial intelligence research.

DeepMind has made significant progress with its PathNet project (Review: DeepMind just published a Mind Blowing paper: PathNet.), a network of neural networks. In another example from the field, just a few days ago, OpenAI launched a transfer learning competition to evaluate the ability of reinforcement learning algorithms to generalize from previous experience, and the algorithms will be evaluated across 30 SEGA “old school” video games.

Recursive cortical networks (RCN) are another promising approach. Developed by Silicon Valley startup Vicarious, RCN was recently used to solve the text-based CAPTCHA test (the Turing Test that automatically distinguishes computers from humans) with high accuracy, using more than 300 times less data than its rivals in the context of a scene text recognition benchmark (Paper: Generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs).

With recent technological advances, many approaches have been considered, developed, or re-explored, including Geoffrey Hinton’s capsule networks (CapNets), neural attention models, single-sample learning, Micro-neural computers (DNC), neuroevolution, evolutionary strategies, and more, further demonstrating the explosive energy of AI research.

All of the techniques described so far are mathematical and statistical in nature, and rely on a lot of computing power and data to succeed. While the mere creation and refinement of such algorithms has shown considerable power, a common criticism of these approaches is that machines still cannot start or learn principles. AlphaZero doesn’t know it’s playing a game, or what a game is.

A growing number of ideas in research are rethinking the core principles of ARTIFICIAL intelligence, including how the human brain works, and how the brains of children work. Although originally inspired by the human brain (hence the name “nerves”), neural networks quickly separated from biology. A common example is that back propagation does not have an equivalent mechanism in nature.

Teaching a machine to learn like a child was one of the oldest ideas in AI in the 1950s, in the time of Turing and Minsky, but the idea is gaining ground as the field of artificial intelligence and neuroscience matures.

The intersection between AI and neuroscience is the Workshop on “Normative Computing for Brains and Machines” mentioned earlier. While the two fields are still getting to know each other, it’s clear that some AI thinkers are increasingly paying attention to neuroscience inspired research, including Yann LeCun, the godfather of deep learning (Video: What are the learning principles of newborns?). And Yoshua Bengio (Video: Bridging the Gap between Deep Learning and Neuroscience).

Josh Tenenbaum, a professor of cognitive science and computing at THE Massachusetts Institute of Technology, suggests a particularly promising area of research. A key part of Tenenbaum’s work is to focus on building quantitative models of what babies or children learn, rather than what she inherited from evolution, which he calls “intuitive physics” and “intuitive psychology.” His work by language, part of the direction of the bayesian probability to promote progress, combined with a variety of methods, such as sign language knowledge representation, the probability of uncertainty cases reasoning and neural network for pattern recognition (video: “built like humans learn to think and machine”, “building like humans see, learning and thinking machines”).

Although MIT launched an initiative in February called The Quest for Intelligence to “crack the smart code,” combining neuroscience, cognitive science and computer science, all this is still theoretical research in the lab, with real-world and industrial results waiting to be produced.

So how close are we to general artificial Intelligence (AGI)? This high-level experience shows contradictory trends. On the one hand, the pace of innovation is dizzying — many of the developments and stories mentioned in this article (AlphaZero, new version of GAN, Capsule Network, RCN to break CAPTCHA, Google’s second-generation TPU, etc.) have come in the last 12 months, in fact most of them in the last six months only. On the other hand, many AI research teams, while actively pursuing AGI themselves, have gone to great lengths to emphasize that we’re still far from it, perhaps out of fear that the media hype around AI will lead to dashed hopes and another AI nuclear winter.

Whether or not we reach AGI any time soon, it’s clear that AI is becoming powerful and will become even more powerful as it runs on increasingly powerful computers, which raises certain concerns about what will happen if its capabilities are in the wrong hands (human or artificial). Elon Musk created “Do You Trust This Computer?” One of the main reasons for the documentary is that ai doesn’t even need to be hostile to humans or even know what humans are. In its relentless efforts to complete a task, it may harm humans simply because they stand in its way, like a road killer.

Physical damage aside, the progress of artificial intelligence, the risk of a series of more pressing from the big industrial age (logistics, trucks) of the important jobs lost, to completely distort our sense of reality (when the false video and audio can be easily created), it is all we need to think thoroughly.

英文原文 :

https://hackernoon.com/frontier-ai-how-far-are-we-from-artificial-general-intelligence-really-5b13b1ebcd4e 

Please pay attention to the wechat public account “AI Front”, (ID: AI-front)