這是2019年5月14日Hinton在北大做的遠程講座

Abstract

In 2006, there was a resurgence of interest in deep neural networks. This was triggered by the discovery that there was a simple and effective way to pre-train deep networks as generative models of unlabeled data. The pre-trained networks could then be fine-tuned discriminatively to give excellent performance on labeled data. In this lecture, I will describe the pre-training procedure used for Deep Belief Nets and show how it evolved from an earlier training procedure for Boltzmann machines that was theoretically elegant but too inefficient to be practical. I will also show how the pre-training procedure overcame a major practical problem in training densely connected belief nets.

2006年，人們對深部神經網絡的興趣重新抬頭。這是由于發(fā)現(xiàn)有一種簡單而有效的方法將深層網絡預訓練為未標記數(shù)據的生成模型而引發(fā)的。然后，可以對經過預先培訓的網絡進行有區(qū)別的微調，以在標記的數(shù)據上提供出色的性能。在這堂課中，我將描述用于深度信念網的預培訓程序，并展示它是如何從早期的Boltzmann機器培訓程序演變而來的，Boltzmann機器在理論上很優(yōu)雅，但效率太低，不實用。我還將展示訓練前的程序如何克服訓練緊密相連的信念網中的一個重大實際問題。

Biography

Geoffrey Hinton received his PhD in Artificial Intelligence from Edinburgh in 1978. After five years as a faculty member at Carnegie-Mellon he became a fellow of the Canadian Institute for Advanced Research and moved to the Department of Computer Science at the University of Toronto where he is now an Emeritus Distinguished Professor. He is also a Vice President & Engineering Fellow at Google and Chief Scientific Adviser of the Vector Institute.

Geoffrey Hinton was one of the researchers who introduced the backpropagation algorithm and the first to use backpropagation for learning word embeddings. His other contributions to neural network research include Boltzmann machines, distributed representations, time-delay neural nets, mixtures of experts, variational learning and deep learning. His research group in Toronto made major breakthroughs in deep learning that revolutionized speech recognition and object classification.

Geoffrey Hinton is a fellow of the UK Royal Society, a foreign member of the US National Academy of Engineering and a foreign member of the American Academy of Arts and Sciences. His awards include the David E. Rumelhart prize, the IJCAI award for research excellence, the Killam prize for Engineering, the IEEE Frank Rosenblatt medal, the IEEE James Clerk Maxwell Gold medal, the NEC C&C award, the BBVA award, the Turing award, and the NSERC Herzberg Gold Medal which is Canada's top award in Science and Engineering.

杰弗里·辛頓于1978年從愛丁堡獲得了人工智能博士學位。在卡內基梅隆大學當了五年的教員后，他成為了加拿大高級研究所的研究員，并搬到了多倫多大學計算機科學系，現(xiàn)在他是一位退休的杰出教授。他還是谷歌的副總裁兼工程研究員，也是矢量研究所的首席科學顧問。

杰弗里·辛頓（geoffrey hinton）是研究者之一，他引入了反向傳播算法，并首次使用反向傳播來學習單詞嵌入。他對神經網絡研究的其他貢獻包括玻爾茲曼機器、分布式表示、時滯神經網絡、專家混合、變分學習和深度學習。他在多倫多的研究小組在深入學習方面取得了重大突破，徹底改變了語音識別和對象分類。

杰弗里·辛頓是英國皇家學會會員，美國國家工程院外籍院士，美國藝術與科學院外籍院士。他的獎項包括大衛(wèi)E.魯梅爾哈特獎、ijcai杰出研究獎、基拉姆工程獎、IEEE Frank Rosenblatt獎、IEEE James Clerk Maxwell金獎、NEC C C&C獎、BBVA獎、圖靈獎和加拿大科學與工程最高獎NSERC Herzberg金獎。