Computational Sensorimotor Learning

Course Description

This course will discuss a working definition of intelligence and use it to delve into the questions of communicating goals and building intelligent agents. We will cover fundamental algorithmic approaches and provide hands on experience for constructing agents that learn to act from raw sensory observations. Topics include imitation learning, observation learning, self-supervised learning, reinforcement learning, inverse reinforcement learning and model learning.

Course Format

This is a graduate course that will be a mix of seminar and lecture style classes. The plan is delve into technical details of particular algorithmic topics and follow it up with reading research papers. The course will involve working on a research project, three assignments and presenting papers.

Prequisites

Working knowledge of machine learning, deep learning, reinforcement learning is assumed. Experience with deep learning packages such as pyTorch / Tensorflow is assumed. Homeworks will be in python. This is NOT a basic course in RL or Deep Learning or AI. Although, this can be used as a concentration subject in AI.

Enrollment

By instructor's permission. The course is currently oversubscribed and you will be added to the waitlist.

Course Information

---

---

Lectures and Readings

This is a new class and there is no textbook. We will post relevant reading materials. Please refer to the schedule.

Assignments

This is a new class and there is no textbook. We will post relevant reading material.

• HW0
• HW1
• HW2
• HW3

Links and due dates for assignments will be on the schedule.

Submission of Assignments
We'll be using Gradescope for problem set submission and grading. Each problem set is weighted equally. The login code for this class will be posted on Piazza -- please create an account and add yourself to this class using that code only if you are taking this class for credit. Grading will rely on review of the submitted code and writeup. More details will be provided when assignments are released.

Collaboration Policy
Collaboration is encouraged, but the work you submit for assignments is expected to be entirely your own. That is, the writing and code must be yours, and you must fully understand everything that you hand in. Discussing the details of how to solve a problem is fine, but you must write the solution yourself. To avoid plagiarizing, you shouldn't be looking at someone else's solution while you write down your own. If you collaborated significantly (use your own discretion for "significantly") on a problem, list the people you collaborated with next to your solution.

---

Final Projects

The final project will be your opportunity to explore some of the topics introduced in the course more deeply.

Details about the final projects will be announced early in the semester.

---

FAQ

What will assignments be like?
The assignments will consist of implementing state of the art reinforcement learning, imitation learning, and model learning algorithms. Programming assignments will be in Python. All assignments (both hand-written components and programming components) will be graded using Gradescope.

I am not sure if I fit the prerequisites, what should I do?
Please speak to a staff member.

What can I do to best prepare for success in the class?
The course requires familiarity with reinforcement learning, numerical optimization, and machine learning. You should be able to propose a project relevant to these topics. If you would like to prepare as best as possible, here is what we recommend:

• Linear Algebra:
• A strong, intuitive understanding of Linear Algebra will very much help you with this class. To brush up on linear algebra, the content and video lectures from Gilbert Strang's classic course, MIT 18.06 have helped many students in the past.
• Another good resource is the textbook Linear Algebra Done Right by Sheldon Axler.
• Programming in Python
• If you are somewhat familiar with Python but would like to brush up on syntax, this tutorial from Stanford CS231n provides a good overview.
• If you are an absolute beginner with Python, Codecademy provides a friendly introduction.
• Mathematical Optimization
• We will teach you what you need to know about optimization, but acquiring a background in the subject will help you deepen understanding.
• Video lectures on Convex Optimization by Stephen Boyd are recommended, as is the reference textbook Convex Optimization by Boyd and Vandenberghe.
• Machine Learning
• Basic background in Machine Learning will help. 6.036 is a good introduction. There are also many great introductory classes online, Andrew Ng's is one.
• Reinforcement Learning
• Basic background in Reinforcement Learning will help. Sutton & Barto, Reinforcement Learning: An Introduction is a good introduction.

---

People

You can contact the course staff at: csl-staff [AT] mit [DOT] edu