• SDS Portal
Search
CUHK-Shenzhen
简体中文
  • About SDS
    • Overview
    • Academic Area
    • Dean’s Message
    • Publications
      • Brochure
      • School Newsletter
      • Annual Report
    • FAQ
    • Contact Us
  • Programmes
    • Introduction
    • Undergraduate
      • Data Science and Big Data Technology
      • Statistics
      • Computer Science and Engineering
      • Financial Engineering
      • 2+2 Double Major Programme
        • Interdisciplinary Data Analytics + X Double Major Programme
        • Aerospace Science and Earth Informatics + X Double Major Programme
    • Taught Postgraduate
      • M.Sc in Data Science
      • M.Sc in Financial Engineering(Full-time/Part-time)
      • M.Sc in Artificial Intelligence and Robotics
      • M.Sc in Computer Science
      • M.Sc in Statistics
      • M.Sc in Bioinformatics
    • Research Postgraduate
      • M.Phil.-Ph.D. Programme in Data Science
      • M.Phil.-Ph.D. Programme in Computer Science
  • Faculty
    • Faculty
    • Emeritus Faculty
    • Affiliated Appointments
    • Researchers/Visitors
  • Students
    • Ph.D. Students
    • Student Interviews
  • News & Announcements
    • News
    • Announcements
  • School Events
    • Academic Conferences
      • ICSR+InnoBiz 2024
      • CSAMSE 2023
      • RMTA 2023
      • ICASSP 2022
      • Mostly OM 2019
    • Academic Activities
    • SDS Colloquium Series
    • Other Events
  • Research
  • Jobs
    • Faculty Positions
    • Postdoctoral Fellowships
  • Career
    • Graduate Placements
    • International Programmes
  • About SDS
    • Overview
    • Academic Area
    • Dean’s Message
    • Publications
      • Brochure
      • School Newsletter
      • Annual Report
    • FAQ
    • Contact Us
  • Programmes
    • Introduction
    • Undergraduate
      • Data Science and Big Data Technology
      • Statistics
      • Computer Science and Engineering
      • Financial Engineering
      • 2+2 Double Major Programme
        • Interdisciplinary Data Analytics + X Double Major Programme
        • Aerospace Science and Earth Informatics + X Double Major Programme
    • Taught Postgraduate
      • M.Sc in Data Science
      • M.Sc in Financial Engineering(Full-time/Part-time)
      • M.Sc in Artificial Intelligence and Robotics
      • M.Sc in Computer Science
      • M.Sc in Statistics
      • M.Sc in Bioinformatics
    • Research Postgraduate
      • M.Phil.-Ph.D. Programme in Data Science
      • M.Phil.-Ph.D. Programme in Computer Science
  • Faculty
    • Faculty
    • Emeritus Faculty
    • Affiliated Appointments
    • Researchers/Visitors
  • Students
    • Ph.D. Students
    • Student Interviews
  • News & Announcements
    • News
    • Announcements
  • School Events
    • Academic Conferences
      • ICSR+InnoBiz 2024
      • CSAMSE 2023
      • RMTA 2023
      • ICASSP 2022
      • Mostly OM 2019
    • Academic Activities
    • SDS Colloquium Series
    • Other Events
  • Research
  • Jobs
    • Faculty Positions
    • Postdoctoral Fellowships
  • Career
    • Graduate Placements
    • International Programmes
  • SDS Portal
CUHK-Shenzhen
简体中文

Breadcrumb

  • Home
  • School Events
  • SDS Colloquium Series
  • 【SDS Colloquium Series】The Increasing Role of Sensorimotor Experience in Artificial Intelligence

【SDS Colloquium Series】The Increasing Role of Sensorimotor Experience in Artificial Intelligence

December 26, 2024 SDS Colloquium Series

SDS Colloquium Series

TopicThe Increasing Role of Sensorimotor Experience in Artificial Intelligence
SpeakerRichard S. SUTTON, Professor, Department of Computing Science, University of Alberta
HostChenjun XIAO, Assistant Professor, School of Data Science, CUHK-Shenzhen
Date26 December (Thursday), 2024
Time3:00 PM - 4:00 PM, Beijing Time
FormatOnsite
VenueRoom 101, Teaching Complex C
LanguageEnglish

Abstract

We receive information about the world through our sensors and influence the world through our effectors. Such low-level experiential data has gradually come to play a greater role in AI during its 70-year history. I see this as occurring in four steps, two of which are mostly past and two of which are in progress or yet to come. The first step was to view AI as the design of agents which interact with the world and thereby have sensorimotor experience; this viewpoint became prominent in the 1990s. The second step was to view the goal of intelligence in terms of experience, as in the reward signal of optimal control and reinforcement learning. The reward formulation of goals is now widely used but rarely loved. Many would prefer to express goals in non-experiential terms, such as reaching a destination or benefiting humanity, but settle for reward because, as an experiential signal, reward is directly available to the agent without human assistance or interpretation. This is the pattern that we see in all four steps. Initially a non-experiential approach seems more intuitive, is preferred and tried, but ultimately proves a limitation on scaling; the experiential approach is more suited to learning and scaling with computational resources. The third step in the increasing role of experience in AI concerns the agent’s representation of the world’s state. Classically, the state of the world is represented in objective terms external to the agent, such as “the grass is wet” and “the car is ten meters in front of me”, or with probability distributions over world states such as in POMDPs and other Bayesian approaches. Alternatively, the state of the world can be represented experientially in terms of summaries of past experience (e.g., the last four Atari video frames input to DQN) or predictions of future experience (e.g., successor representations). The fourth step is potentially the biggest: world knowledge. Classically, world knowledge has always been expressed in terms far from experience, and this has limited its ability to be learned and maintained. Today we are seeing more calls for knowledge to be predictive and grounded in experience. If completed, the four steps might enable substantial new progress in AI.

Biography

Richard S. Sutton is a professor in the Department of Computing Science at the University of Alberta, a Canadian AI Chair, and a fellow of the Royal Society of London, the Royal Society of Canada, the Association for the Advancement of Artificial Intelligence, the Alberta Machine Intelligence Institute (Amii), and CIFAR. He is also a Research Scientist at Keen Technologies and the Founder of the Openmind Research Institute. He received a PhD in computer science from the University of Massachusetts in 1984 and a BA in psychology from Stanford University in 1978.

 

Prior to joining the University of Alberta in 2003, he worked in industry at AT&T Labs and GTE Labs, and in academia at the University of Massachusetts. At the University of Alberta, Sutton founded the Reinforcement Learning and Artificial Intelligence Lab. From 2017-2023 he founded and worked at DeepMind Alberta in Edmonton. Sutton’s research interests center on the learning problems facing a decision-making agent interacting with its environment, which he sees as central to intelligence. He has additional interests in animal learning psychology, in connectionist networks, and generally in systems that continually improve their representations and models of the world. He is co-author of the textbook Reinforcement Learning: An Introduction. His scientific publications have been cited more than 150,000 times. He is also a libertarian, a chess player, and a cancer survivor.

Address: 3 - 6 Floor, Dao Yuan Building, 2001 Longxiang Road, Longgang District, Shenzhen
E-mail: sds@cuhk.edu.cn
Wechat Account: cuhksz-sds

sds.cuhk.edu.cn

Copyright © CUHK-Shenzhen School of Data Science