In this lab you will familiarize yourself with pyro, a tool for controlling both simulated and physical robots, that is written in python. All of the source code for pyro is located at /usr/local/pyrobot/. Feel free to go check out any aspect that interests you. Pyro includes many machine learning tools, such as neural networks, which we will explore this week. To begin, run update81 to copy some starting point files into your home directory (cs81/labs/1/).

I have provided three examples of simple neural networks for solving the logic problems and, or, and xor. Open one of these files in an editor and read through the code. Then execute it in python. Notice that only two layers of units are necessary to solve and and or, but three layers of units are necessary to solve xor.

• Which of one three problems tends to find a solution in the fewest number of epochs (one run through the entire data set)?
• Try experimenting with the learning rate (called epsilon). Set it very low (at say 0.1) and set it very high (at say 1.0). How does the speed of learning change?
• Try examining the weights after learning has been completed. Train the network using the -i flag (for interactive):

  % python -i and.py
  

  This will execute the code and then leave you in the python interpreter. Do the following to see the weights:

  >>> n.getWeights("input", "output")
  

  Do the following to see the bias of a particular node:

  >>> n["output"].weight[0]
  

Draw a picture of the network with the weights and the biases and try to understand how the solution operates.

One method for teaching a neural network to control a robot is called offline training and involves the following steps. First hand code a robot controller. Second use this controller to collect data to be used to train a neural network. Third train a neural network with the collected data. Fourth use the trained network to control the robot and test its performance.

• I have provided a hand coded program called wallFollow.py that causes the robot to follow walls on its left hand side. To observe a robot using this program do the following:

  %pyrobot -s PyrobotSimulator -w Tutorial.py -r PyrobotRobot60000 -b wallFollow.py
  

  The -s flag specifies which simulator you would like to use. Each simulator has a number of different worlds, which are specified by the -w flag. The simulator can control multiple robots simultaneously. Each robot has a name specified by the -r flag. Finally the robot is controlled by a brain, which is specified by the -b flag.

  This will open two windows: one the pyro control window and the other the simulator window. In the simulator window, select the "View" menu and choose "Trail", to see the robot's path. In the pyro window, push the "Run" button to start the robot. Every 500 steps the program will report the average distance of the robot from the wall on its left side. We will use data collected from a program running a controller like this to train a neural network.

  To quit, first press the "Stop" button in the pyro window. Then, in the pyro window, go to the "File" menu and select "Exit". If the pyro window fails to close, then in a terminal window do: killall -9 pyrobot.

• Open the wallFollow.py program in an editor. Notice that it contains a class that inherits from a Brain class. In pyro all controllers are constructed in this manner. Each brain can optionally use the setup method for initialization. It will be run once when the object is instantiated. Each brain must have a step method. This is the what will be executed on each time step (typically about 10 times a second). The move command is what causes the robot to move. It takes two parameters: translation (forward/backward) and rotation (left/right). Each of these parameters can be in the range -1 to +1.
• We will use the program collectData.py to execute the wall following behavior and collect data along the way. It is important to remember that all data used with a neural network must be scaled to match the range of the activation function. In our case, all data must be scaled between 0 and 1. To run the data collection program do:

  %pyrobot -s PyrobotSimulator -w Tutorial.py -r PyrobotRobot60000 -b collectData.py

  When you see a message that the data collection is done. Stop the robot and exit from pyro. You should now have two files called sensorInputs.dat and motorOutputs.dat saved in your directory.
• Next we will use the program trainNetwork.py to train the neural network on the saved data. We do not need to use the simulator to do this part.

  %python trainNetwork.py
  

  This will take several minutes to complete. It will execute 1000 epochs of training on the collected data. Watch the error and percent correct during training. Does error consistently go down? Does percent correct consistently go up? After training is complete, you will have a new file in your directory called wallFollow.wts containing the weights and biases of the most successful version of the trained neural network.
• Finally, we can use the saved weights to see how well the network controls the robot.

   
  %pyrobot -s PyrobotSimulator -w Tutorial.py -r PyrobotRobot60000 -b testNetwork.py

  Be sure to "View" the robot's "Trail" in the simulator window before pressing the "Run" button in the pyro window. How does the neural network controlled robot perform compared to the hand-coded teacher program? In what ways does the neural network controlled robot behave differently from the teacher?

You will go through the same series of steps as above, but this time using your own hand-coded teacher program. Put all of the files in the cs81/labs/1 directory. Also create a text summary file in this directory to describe what your teacher program does, how you trained the network, and your results.

---

When you are done, run handin81 to turn in your completed lab work.