Class information

Class: T, TH 1:15–2:30 SCI L26
Lab A: M 1:15–2:45 SCI 256
Lab B: M 3:00–4:30 SCI 256

Professor: Lisa Meeden
Office: Science Center 243
Phone: 328-8565

Introduction

This seminar will examine ways of making robots be more adaptive. We will investigate methods that allow robots to learn about themselves and their environment by autonomously exploring what they can achieve rather than being told how to behave. We will focus on machine learning approaches including evolutionary robotics, developmental robotics, unsupervised learning, reinforcement learning, and deep learning. This is a discussion-based course that relies on students doing a close reading of assigned research papers and coming to class prepared to actively engage with the material.

Goals for the course

• Analyze and critically discuss technical, research papers both in writing and in class.
• Understand the fundamental questions in a research field and be able to compare and contrast different approaches to answering these questions.
• Formulate and evaluate a research question in adaptive robotics by completing a substantial project.
• Relate your project to prior research via a review of related literature.
• Visually present a clear and accessible summary of a research project using a poster.
• Orally present a clear and accessible summary of a research project.
• Write a coherent conference-style paper describing and evaluating a research project.

Grading

• Class Participation and Paper Responses: 25%
  Each week we will discuss papers in a seminar style. This is not a lecture-based course. The class as a whole will generate the course content, and you need to be present to contribute. Doing a close reading of the assigned papers and writing responses and questions prior to each class is essential to preparing for a lively and informed discussion.

• Labs: 30%
  During the first half of the semester we will focus on implementing and experimenting with the models that we are reading about in the primary literature. In the second half of the semester we will use lab time to make progress on the project.

• Project: 45%
  You will design a project related to adaptive robotics. You are strongly encouraged to work with a partner.
• Proposal 5%
  
• Checkpoint demonstration 5%
  
• Presentation 10%
  
• Paper 25%
  

Accommodations

If you believe that you need accommodations for a disability, please contact the Office of Student Disability Services (Parrish 113W) or email studentdisabilityservices@swarthmore.edu to arrange an appointment to discuss your needs. As appropriate, the Office will issue students with documented disabilities a formal Accommodations Letter. Since accommodations require early planning and are not retroactive, please contact the Office of Student Disability Services as soon as possible. For details about the accommodations process, visit the Student Disability Service Website.

You are also welcome to contact me privately to discuss your academic needs. However, all disability-related accommodations must be arranged through the Office of Student Disability Services.

Paper Responses

Each class meeting will typically focus on one paper. Prepare for class by reading the assigned paper and writing a response in the format described below.

• Your response must be typed (in 12-point font) and be no longer than a single page. You will upload a PDF file with your response using Moodle by 10am on the day the paper is scheduled for discussion. I will review and comment on your responses before class.
• Your response should provide an overview that describes the paper's main points. The papers will be too long to give a complete summary. Instead you should focus on the main themes and overall conclusions. The challenging aspect of these responses is being able to distill the essential information in a succinct way.
• Your response should also include three questions or issues that we can discuss during class. Make sure you have access to your response during class time, either by bringing a hard copy or by viewing a digital version.
  
  These questions should include big picture issues, such as:
  • In what ways does the mechanism described seem like a good tool for doing adaptive robotics? What are its advantages/disadvantages?
  • Do the paper's conclusions logically follow from the data provided?
  • How do the paper's mechanisms and conclusions relate to previous papers that we have read?
  • If the paper describes an experiment, what follow up experiments should be done and why?
  These questions might also include requests for clarification, such as:
  • What is the meaning of some specific terminology? Though you should try to investigate this on your own as well.
  • How does a particular algorithm work? Why is a certain step of an algorithm necessary?
  • What is being shown in a particular figure or table in the paper?

---

Schedule

WEEK	DAY	ANNOUNCEMENTS	TOPIC & READING	LAB
1	Sep 05		Introduction to Adaptive Robotics and Neural Networks Evolutionary Robotics Josh Bongard, CACM, Vol. 56, No. 8 (2013) What is Deep Learning? Francois Chollet, Ch. 1 from Deep Learning with Python (2017) Softbot evolution video	1: Simulating Robots in Jupyter Notebooks
	Sep 07
2	Sep 12		Developmental Robotics Some Basic Principles of Developmental Robotics Alexander Stoytchev, IEEE Transactions on Autonomous Mental Development, Vol. 1, No. 2 (2009) Developing Grounded Goals through Instant Replay Learning Lisa Meeden and Douglas Blank, Proceedings of Conference on Development and Learning and on Epigenetic Robotics (2017) Video of baby exploring its environment	2: Neural Network Robot Controllers
	Sep 14
3	Sep 19	Bryce Wiedenbeck Guest lecturer	Human-level control through deep reinforcement learning V. Minh et. al., Nature, Vol. 518 (2015) Kinds of Minds: Toward an Understanding of Consciousness Daniel Dennett, Chs. 3-4 (1996)	No lab Lisa attending EpiRob17 Conference
	Sep 21	Frank Durgin Guest lecturer
4	Sep 26	Read Sections 1-3	Evolving Neural Networks Competitive coevolution through evolutionary complexification Kenneth Stanley and Miikkulainen, Journal of AI Research, Vol. 21 (2004) Robot Duel Demos	3: Evolving Neural Network Controllers
	Sep 28	Read Sections 4-7
5	Oct 03		Novelty-based vs Objective-based Evolution Abandoning Objectives: Evolution through the Search for Novelty Alone Joel Lehman and Kenneth Stanley, Evolutionary Computation, Vol. 19, No. 2 (2011) Evolving a Diversity of Creatures through Novelty Search and Local Competition Joel Lehman and Kenneth Stanley, Proceedings of the GECCO Conference (2011) Novelty Search Video	4: Implementing Novelty Search
	Oct 05
6	Oct 10		Active Vision Image kernels explained visually A beginner's guide to understanding convolutional neural networks Part 1, Part 2 Active grounding of visual situations Max Quinn, Erik Conser, Jordan Witte, Melanie Mitchell, Unpublished draft	5: Applying Novelty Search
	Oct 12	Melanie Mitchell Guest lecturer
	Oct 17	Fall Break
	Oct 19
7	Oct 24		Curiosity-Driven Learning How evolution may work through curiosity-driven developmental process, Pierre-Yves Oudeyer and Linda B. Smith, Topics in Cognitive Science (2016) An egocentric perspective on active vision and visual object learning in toddlers, Sven Bambach, David Crandall, Linda B. Smith, and Yu Chen, Proceedings of Conference on Development and Learning and on Epigenetic Robotics (2017)	6: Creating and Presenting a Research Poster
	Oct 26
8	Oct 31		Deep Learning Deep Learning, LeCun, Bengio, and Hinton, in Nature, Vol. 521 (2015) A neural algorithm of artistic style, Gatys, Ecker, and Bethge (2015) Leon Gatys' presentation of artistic style paper	Poster rubric Project
	Nov 02
9	Nov 07		Reinforcement Learning A brief survey of deep reinforcement learning, Arulkumaran, Deisenroth, Brundage, and Bharath in IEEE Signal Processing Magazine (2017) Learning to navigate in complex environments, DeepMind (2017) Understanding LSTM networks	Discuss Project Proposal
	Nov 09
10	Nov 14		Unsupervised Learning Building high-level features using large scale unsupervised learning, Quoc V. Le, et al. in Proceedings of Conference on Machine Learning (2012)	Continue Project
	Nov 16
11	Nov 21	Demos schedule	Demo rubric Checkpoint Demos (day 2)	Checkpoint Demos (day 1)
	Nov 23	Thanksgiving
12	Nov 28		Continue Project (meet in lab)	Continue Project
	Nov 30	Presentation schedule	Presentation rubric Project Presentations (day 1)
13	Dec 05		Project Presentations (day 3)	Project Presentations (day 2)
	Dec 07		Project Presentations (day 4)
14	Dec 12		Project Presentations (day 5)	No lab

Final Papers

• Luis Ceballos and Sarah Wallace
  Facial Expression Recognition Using CNNs
• Henry Feinstein and Eric Wang
  Improving Q-Learning with Neural Networks
• Dina Ginzburg
  Generating Globally Coherent Original Melodies using LSTMs and music21
• Yosuke Higashi and Michael Song
  Active Learning for End-to-End Autonomous Driving
• Nhung Hoang and Ellen Liu
  Transfer Learning in a Jigsaw Puzzle-Like Image Similarity Task
• Connor Keane
  Evolving Robust Robots
• Oliver Lipton and Harry Huchra
  Locality through Modular Network Topology
• Alexander Mandel and Ivan Lomeli
  Automating Graphic Design
• Gabriel Meyer-Lee, Harsha Uppili, and Alan Zhuolun Zhao
  Comparing Evolutionary Algorithms for Deep Neural Networks
• Do June Min
  Using NEAT + ES to Play Games
• Timothy Nguyen and Kastan Day
  Facial Recognition with Information Maximizing Generative Adversarial Networks
• Jeff Novak and Kyle Richmond-Crosset
  Evolutionary Algorithms using Combined Objective and Novelty Search in Unconstrained Space
• Joshua Powell
  Pedestrian Detection with Convolutional Neural Networks
• Alison Rosenzweig and Emma Remy
  Implementing NEAT with a Scribbler Robot
• Becky Tang and Alex Robey
  Autonomous Robotic Navigation using Self-Organized Maps
• Annie Zhao and Owen Kephart
  Comparing Inputs to Evolutionary Algorithms for Maze Pathfinding