Class information

Class: T, TH 9:55–11:10am SCI 181
Lab A: F 2:15–3:45 SCI 252
Lab B: F 4:00–5:30 SCI 252

Professor: Lisa Meeden
Office: Science Center 243
Phone: 328-8565

Introduction

This seminar will explore the topic of adaptive robotics with a focus on developmental robotics, a newly emerging paradigm of research. The goal of developmental robotics is to create intelligent robots by allowing them to go through a developmental process, rather than being directly programmed to solve a particular task. By endowing a robot with an appropriate initial control architecture and adaptive mechanisms, it can learn through interactions with the world, developing self-organized knowledge about itself and its environment. We will be studying the following sorts of questions: What should be innate in the robot? What adaptive mechanisms are needed? What motivates the robot to act?

Goals for the course

• Learn how to test, debug, and verify the correctness of programs that implement machine learning algorithms.
• Learn how to think like a computer science researcher:
• Analyze and critically discuss technical, research papers both in writing and in class.
• Understand the fundamental questions in a research field and be able to analyze different approaches to answering these questions.
• Formulate and evaluate a research question in a technical field by completing a substantial project.
• Relate your project to prior research via a review of related literature.
• Orally present a clear and accessible summary of your project.
• Write a coherent conference-style paper describing and evaluating your project.

Grading

• Class Participation and Paper Responses: 25%
  Each week we will discuss papers in a seminar style. This is not a lecture-based course. The class as a whole will generate the course content, and you need to be present to contribute. Doing a close reading of the assigned papers and writing responses and questions prior to each class is essential to preparing for a lively and informed discussion.

• Labs: 30%
  During the first half of the semester we will focus on implementing and experimenting with the models that we are reading about in the primary literature. In the second half of the semester we will use lab time to make progress on the project.

• Project: 45%
  You will design a project related to adaptive robotics. You are strongly encouraged to work with a partner.
• Proposal 10%
  
• Checkpoint demonstration 5%
  
• Presentation 10%
  
• Paper 20%
  

Paper Responses

Each class meeting will focus on one paper. Prepare for class by reading the assigned paper and writing a brief response (no more than one page in length). Your response should be typed and printed, and will be handed in at the end of class. Your response should include:

• A one paragraph overview that describes the paper's main points.
• At least three questions or issues that we can discuss during class. These might include questions of clarification, such as:
  • What is the meaning of some specific terminology? Though you should try to investigate this on your own as well.
  • How does a particular algorithm work? Why is a certain step of an algorithm necessary?
  • Clarify what is being shown in a particular figure or table in the paper.
  But should also include bigger picture issues, such as:
  • In what ways does the mechanism described seem like a good tool for doing adaptive robotics? What are its advantages/disadvantages?
  • Do the paper's conclusions logically follow from the data provided?
  • How do the paper's mechanisms and conclusions relate to previous papers that we have read?
  • What follow up experiments should be done and why?

---

Schedule

WEEK	DAY	ANNOUNCEMENTS	TOPIC & READING	LAB
1	Sep 01		Introduction Growing Babies and Robots Ch. 1 from Developmental Robotics, Cangelosi and Schlesinger (2015) [Read for Thursday]	1: Scribbler Robot
	Sep 03
2	Sep 08		Neural Networks Chapters 1-2 from Computational Developmental Psychology, Thomas Schultz (1995) [Read Ch. 1 for Tuesday and Ch. 2 for Thursday] Derivation of Back Propagation	2: Robot Simulator
	Sep 10	Drop/Add ends (Sep 11)
3	Sep 15		Evolving Neural Networks Genetic Algorithms: An Overview Ch. 1 (pg 2-10) from Mitchell (1999) Competitive coevolution through evolutionary complexification Journal of Artificial Intelligence Research, Vol. 21, Stanley and Miikkulainen (2004) Robot duel demo	3: Evolving neural network controllers
	Sep 17
4	Sep 22		Novelty search Abandoning objectives: Evolution through the search for novelty alone Evolutionary Computation, Vol. 19:2, Lehman and Stanley (2011) Evolving a diversity of creatures through novelty search and local competition In the Proceedings of the Genetic and Evolutionary Computation Conference, Lehman and Stanley (2011) Novelty video	4: Implementing Novelty Search
	Sep 24
5	Sep 29		Developmental Encodings Compositional pattern producing networks: A novel abstraction of development, by Kenneth O. Stanley, in Genetic programming and evolvable machines, Vol. 8, (2007) Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding by Cheney, MacCurdy, Clune, and Lipson, in Proceedings of GECCO (2013) Softbot video	5: Applying Novelty Search
	Oct 01
6	Oct 06		Intrinsic Motivation Intrinsic motivation systems for autonomous mental development IEEE Transactions on Evolutionary Computation, vol. 11.2, Oudeyer, Kaplan, and Hafner (2007) [Read Sections 1-6 for Tuesday, and the remainder for Thursday]	None
	Oct 08
	Oct 13	Fall Break
	Oct 15
7	Oct 20		Unsupervised learning A growing neural gas learns topologies Advances in Neural Information Processing Systems 7, Bernd Fritzke (1995). Category-based intrinsic motivation, Proceedings of the Ninth International Conference on Epigenetic Robotics, Lee, Walker, Meeden, and Marshall (2009)	Project
	Oct 22
8	Oct 27		Reinforcement learning Reinforcement learning: An introduction, Richard Sutton and Andrew Barto, MIT Press (1998), Chapter 1: Introduction (through section 1.5) Learning to see and act, Scholkopf in Nature, Volume 518 (2015) Human-level control through deep reinforcement learning, Mnih et. al. in Nature, Volume 518 (2015)	Discuss project proposal
	Oct 29	Project proposal due
9	Nov 03		Cooperation Coevolution of role-based cooperation in multiagent systems Yong and Miikkulainen, in IEEE Transactions on Autonomus Mental Development, Volume 1, Number 3 (2009)	Continue project
	Nov 05	Last day to declare CR/NC (Nov 06)
10	Nov 10		Deep learning Convolutional Neural Networks Notes from Karpathy's Stanford course CS231n Scaling learning algorithms towards AI Bengio and LeCun (2007), Focus on sections 1-2 and 5-7	Prepare for checkpoint
	Nov 12
11	Nov 17		Checkpoint demonstration schedule Rubric for checkpoint demonstration	Prepare for presentation and begin paper
	Nov 19
12	Nov 24		CS Research Talks Example presentation: Slides from EpiRob09 Talk on the paper Category-based intrinsic motivation Tips for preparing a research presentation Rubric for presentations	None
	Nov 26	Thanksgiving
13	Dec 01	Project work day in lab	Project presentation schedule
	Dec 03
14	Dec 08	Party!

Final Papers

• Amy Han and Jeremy Han, Evolving Mario to Maximize Coin Score using NEAT and Novelty
• Andy Lee, Comparing Deep Neural Networks and Traditional Vision Algorithms in Mobile Robotics
• Angela Mastrianni and Lucas Franca do Nascimento Farias, Using NEAT to Evolve Cooperation and Intelligence in Game Theory Problems
• Ben Schreiber and Yikai Wang, Creating a Traffic Merging Behavior using NeuroEvolution of Augmenting Topologies
• Billy Evers and Mike Rubayo, Competition Between Objective and Novelty Search on a Deceptive Task
• Connor Cassidy, Unshackling Unshackling Evolution: Evolving Softbots with a Multitude of Materials
• Erin Gluck and Evelyn Wightman, SimAnt Simulation using NEAT
• George Abraham and Josh Wolper, Evolving Novel Cellular Automaton Seeds using Computational Pattern Producing Networks
• Julian Segert, Crawling Before Walking: Improving Walking Soft Robot Evolution
• Maddox Riley, Measuring NEAT Performance on a Variable Deceptive Bipedal Walking Task
• Maria-Elena Solano, Evolution with Culture
• Matt Baer, Co-Evolution of Multi-Agent Systems using NEAT
• Rex Chang and Sedi Worlanyo, Evolving Swarm Communication with NEAT
• Richard Phillips, Generating Near-Optimized Molecular Geometries Across Reactions using Neural Networks and Back Propagation
• Tahmid Rahman and Shantanu Jain, Evolving Cooperative Communicative Behavior