【数据科学名家讲坛】The Increasing Role of Sensorimotor Experience in Artificial Intelligence
SDS Colloquium Series | |
| Topic | The Increasing Role of Sensorimotor Experience in Artificial Intelligence |
| Speaker | Richard S. SUTTON, Professor, Department of Computing Science, University of Alberta |
| Host | Chenjun XIAO, Assistant Professor, School of Data Science, CUHK-Shenzhen |
| Date | 26 December (Thursday), 2024 |
| Time | 3:00 PM - 4:00 PM, Beijing Time |
| Format | Onsite |
| Venue | Room 101, Teaching Complex C |
| Language | English |
Abstract | |
| We receive information about the world through our sensors and influence the world through our effectors. Such low-level experiential data has gradually come to play a greater role in AI during its 70-year history. I see this as occurring in four steps, two of which are mostly past and two of which are in progress or yet to come. The first step was to view AI as the design of agents which interact with the world and thereby have sensorimotor experience; this viewpoint became prominent in the 1990s. The second step was to view the goal of intelligence in terms of experience, as in the reward signal of optimal control and reinforcement learning. The reward formulation of goals is now widely used but rarely loved. Many would prefer to express goals in non-experiential terms, such as reaching a destination or benefiting humanity, but settle for reward because, as an experiential signal, reward is directly available to the agent without human assistance or interpretation. This is the pattern that we see in all four steps. Initially a non-experiential approach seems more intuitive, is preferred and tried, but ultimately proves a limitation on scaling; the experiential approach is more suited to learning and scaling with computational resources. The third step in the increasing role of experience in AI concerns the agent’s representation of the world’s state. Classically, the state of the world is represented in objective terms external to the agent, such as “the grass is wet” and “the car is ten meters in front of me”, or with probability distributions over world states such as in POMDPs and other Bayesian approaches. Alternatively, the state of the world can be represented experientially in terms of summaries of past experience (e.g., the last four Atari video frames input to DQN) or predictions of future experience (e.g., successor representations). The fourth step is potentially the biggest: world knowledge. Classically, world knowledge has always been expressed in terms far from experience, and this has limited its ability to be learned and maintained. Today we are seeing more calls for knowledge to be predictive and grounded in experience. If completed, the four steps might enable substantial new progress in AI. | |
Biography | |
Richard S. Sutton is a professor in the Department of Computing Science at the University of Alberta, a Canadian AI Chair, and a fellow of the Royal Society of London, the Royal Society of Canada, the Association for the Advancement of Artificial Intelligence, the Alberta Machine Intelligence Institute (Amii), and CIFAR. He is also a Research Scientist at Keen Technologies and the Founder of the Openmind Research Institute. He received a PhD in computer science from the University of Massachusetts in 1984 and a BA in psychology from Stanford University in 1978.
Prior to joining the University of Alberta in 2003, he worked in industry at AT&T Labs and GTE Labs, and in academia at the University of Massachusetts. At the University of Alberta, Sutton founded the Reinforcement Learning and Artificial Intelligence Lab. From 2017-2023 he founded and worked at DeepMind Alberta in Edmonton. Sutton’s research interests center on the learning problems facing a decision-making agent interacting with its environment, which he sees as central to intelligence. He has additional interests in animal learning psychology, in connectionist networks, and generally in systems that continually improve their representations and models of the world. He is co-author of the textbook Reinforcement Learning: An Introduction. His scientific publications have been cited more than 150,000 times. He is also a libertarian, a chess player, and a cancer survivor. | |
