CS 91 Lab 2

Software-defined networking / OpenFlow

All features due 11:59pm, Thursday, October 2

Handy References

• POX documentation.
• NetworkX documentation. Of particular interest, the tutorial and all_shortest_paths() function.
• Mininet documentation, including a thorough walkthrough.

Lab 2 Goals:

• Analyze and solve datacenter networking problems in the context of software-defined networking.
• Gain hands-on experience with network topologies and their implications for routing, particularly in the presence of multiple paths equal-cost paths.
• Write programs to take advantage of OpenFlow's capabilities using the popular Pox controller.
• Utilize a virtual machine to simulate a complex network environment.

Lab Description

This will be a two-week lab with two parts. First you will establish shortest-path connectivity between hosts across a network topology with multiple equal-cost paths. Next, you will add some packet inspection and modification functionality to behave like a basic firewall.

Environment

For this lab, we'll be using Mininet to simulate small datacenter network architectures like the ones we've been reading about in class. Because Mininet requires a very specific software configuration, we'll be running it inside of a VM to make things easier.

Due to storage space constraints, you'll need to store your VM images on /local rather than your home directory. Storing things on /local has two major implications that you need to account for:

1. The data on /local is stored on a single disk, unlike your home directory, which is split across multiple disks for redundancy. When you're done working, you should save your important lab files elsewhere to avoid data loss in the event of a disk failure. When you're done working on your VM, you should shut it down nicely: "sudo shutdown -h now"
2. The /local partition on each machine is only available on that machine. This means that if you want to move and work a different machine, you'll need to set up a new VM at the new location. You could also copy your VM image from /local on one machine to /local on another with scp.

To get started, you'll need to import a copy of the starter VM image:

1. Run virtualbox in a terminal or open it from within your favorite graphical menu.
2. Go to File->Preferences, set your "Default Machine Folder" to "/local", and then close the preferences window.
3. Go to File->Import Applicance. Choose: /local/CS91-mininet-base.ovf and push next once.
4. You should now be seeing "Appliance settings". Edit the name to include your username. For example, CS91Mininet-VM-kwebb.
5. Click import and wait a minute for it to complete.

After you've completed these steps once, you should see the your VM in the list of VMs available to start. Go ahead and turn it on.

While you could work from within the new VM window that just came up, doing so is a huge pain if you enjoy nice things like graphical interfaces. Instead, it's much easier to connect to the VM via ssh with X11 forwarding turned on. The VM is already configured such that you can connect to it by sshing to your local machine on port 22222. (Port 22222 on your machine gets forwarded to port 22 on the VM):

ssh -Y localhost -p 22222 -l mininet

(The password is: mininet)

Note that you will need two terminals connected to the VM: one to run Mininet, and another to run the Pox OpenFlow controller. You can ssh in from two different terminals, but I recommend trying out the screen utility.

Running Mininet and Pox

Once you're connected to the VM over ssh, you'll need to start Pox, the OpenFlow controller that will tell your switches what to do, and Mininet, the network simulator. It shouldn't matter which one you start first.

When you invoke Mininet, you need to choose a topology. I've provided two example networks: a basic three level, singly-root tree and a k=4 Al-Fares fat tree like the one in figure 3. To start Mininet:

cd ~/lab2

For the simple tree topology:
sudo mn --controller=remote --custom basictree.py --topo basictree

For the k=4 fat tree topology:
sudo mn --controller=remote --custom k4fat.py --topo k4fat

To start Pox:

cd ~/pox

./pox.py openflow.discovery lab2 --hosts=[host file] --firewall=[firewall file]

Where [hosts file] is the file containing host information for the Mininet
topology you chose (basichosts or k4hosts).  The --firewall argument is
optional.  If present, it will define a set of policy rules for your switches
to enforce (part 2).

Once both Mininet and Pox are running, you can issue commands to the virtual hosts. Within Mininet, you can run commands at each host by invoking a terminal, which will pop-up in a new window:

mininet> xterm [host name]

For example:
mininet> xterm host_A1

As a shortcut, you can also issue some commands directly on the Mininet command line. For instance, if you want one host to "ping" another (send it a small message to test connectivity), you can issue:

mininet> host_A1 ping host_A2

Part 1: Getting packets to their destinations.

For your solution, you'll be editing the controller code: lab2.py. This should be the only file you need to submit, unless you choose to divide your code into multiple modules/files (which is fine with me, but not required).

There are three main things going on in lab2.py:

1. At the bottom of the file is a launch function. This is guaranteed to be called before any of your other code executes. Use this to initialize anything that you may need. For example reading host/firewall input files (see below).
2. In the middle of the file is the definition of a Lab2 class. There will be exactly one of these objects created. It represents the state of the lab 2 controller as a whole.
3. Towards the top of the file is the definition of a Lab2Switch class. Objects of this type represent the state of one switch in the topology. There will be a 1:1 correspondence between switches in your topology and object/instances of the Lab2Switch class. These objects have a field named connection, which represents the connection between your controller and the corresponding switch in the network topology. This connection knows the unique identifier of the switch (self.connection.dpid), and it's also what you'll use to send OpenFlow messages to the switch when you want it to do something.

As an input, your controller will receive a "hosts" file that contains connectivity information for each host in the network. You'll use the information it provides to respond to queries about a host's address information (ARP) and determine which destination switch to forward a packet to to reach a target host. This file is a list of the following information, one line per host:

HostName IPAddress MACAddress SwitchDPID SwitchPort

For example:
host_A_A1 10.1.1.1 00:00:00:01:11:11 00-00-00-01-00-01-00-00 1

This entry says that the host named "host_A_A1" has IP address 10.1.1.1, MAC address 00:00:00:01:11:11 and that it's connected to port #1 of the switch whose DPID is 00-00-00-01-00-01-00-00. (A DPID is just a unique identifier for a switch - DataPath IDentifier.)

Network Events: Links becoming (un)available

When something of interest happens in your virtual network (e.g., a new switch comes online, or a packet is received and a switch isn't sure what to do with it), your controller will receive an event notification. For network events (switch arrival/departure), the Lab2 Pox component will receive a call to the _handle_LinkEvent method with information about which link(s) are now (un)available. You'll use this information to maintain a switch connectivity graph so that you can route packets to their destinations.

The NetworkX graph library will take care of most of the heavy lifting for you, including shortest path calculations. Note that for each edge in your graph, you will need to include information about the switch port numbers so that when you'll know which port to send it out of when you go to forward a packet.

You should respond to both link up events (add edge to graph) and link down events (remove edge from graph) to allow routing of new flows to continue, even if a link fails. The starter code provides the details of the event notification format.

Switch Events: Packets to be handled

When a switch receives a packet for which it has no matching flow entry, it will forward that packet to your controller, allowing you to decide its fate. When this occurs, one of your Lab2Switch instances will receive a call to the _handle_PacketIn method with the details of the packet's headers. You can then issue OpenFlow commands to the switch to tell it what to do in response, such as forwarding the packet, installing a flow entry into the switch's table, etc. If no path exists to route the packet, print a message saying so, including some information about the packet.

The event notification also includes other potentially useful information. For example, event.port is the port number of the switch that received the packet. You'll likely find this useful when responding to ARP queries, since those just need to be sent out the port they came in from. This is a simple and effective way of identifying that port number.

You should expect to handle two types of packets, ARP and IPv4. Any other type of packet may be ignored. The way you'll handle them is very different:

• Address Resolution Protocol (ARP): An ARP packet is sent by a host to find out the MAC address of another host. Typically applications (and users) specify IP addresses as the destination for network traffic, so the source will send an ARP request to discover the destination's layer 2/Ethernet MAC address.
  
  Upon receiving an ARP request at the controller, you should already have all the information you need to answer the query due to the details that are provided in the hosts file. You should create an ARP response, containing the correct MAC address for the requested destination, and send it back out the switch port it came in from. Despite being generated by your controller, your ARP response should look as if it was coming from host whose address was queried.
  
  For ARP, you don't need to add any flow entries to switch tables. Simply responding with the correct address is sufficient.


• IP (version 4): IP packets are application data packets that you should route to their final destination. Looking at the packet's destination IP address, use your knowledge of the destination's edge switch (from the hosts file) and your graph of the network to find the shortest path through the network. If there are multiple, equal-length paths, choose one in such way that the same choice will not be made for every flow.
  
  Once you've decided on a path, you should forward the packet along that path and also install a flow table entry into the switch so that all future packets of the flow will not be sent to the controller when they arrive at this switch. Doing so also prevents the controller from making different decisions for each packet of a flow, which could potentially lead to packet reordering.
  
  For this lab, it's fine to add flow table entires as packets arrive along the path one switch at a time rather than installing entries for the entire path at once.

Issuing OpenFlow Directives in POX

The Pox documentation describes the details for issuing OpenFlow commands. As a brief summary, to send a packet:

Create an of.ofp_packet_out() object:

openflow_command = of.ofp_packet_out()

Set the data (here, packet is an Ethernet frame object):

openflow_command.data = packet.pack()

Set one or more OpenFlow actions. In this case, just the "output action": of.ofp_action_output. Fill in the port= with the number of the port you'd like the switch to send it out:

openflow_command.actions.append(of.ofp_action_output(port=...))

After you've set up the openflow command object, you can send it to the switch via a connection to the switch. If you're executing within the context of a Lab2Switch's _handle_PacketIn handler, which is likely, the connection is accessible as self.connection:

self.connection.send(openflow_command)

Installing a flow table entry is very similar, only you create an of.ofp_flow_mod rather than of.ofp_packet_out. You must also set the .match attribute to describe which packets should match this rule. It has the following attributes (and others too):

# Note: This match field is required to be able to match on IP fields in the
# network-layer headers.  If you get "Fields ignored due to unspecified
# prerequisites" warnings, it's because you didn't set this:

openflow_flowmod.match.dl_type = pkt.ethernet.IP_TYPE

openflow_flowmod.match.nw_src  (Source IP address)
openflow_flowmod.match.nw_dst  (Destination IP address)
openflow_flowmod.match.nw_proto  (Layer 4 protocol - ICMP is 1, TCP is 6, UDP is 17)

For TCP/UDP:
openflow_flowmod.match.tp_src  (Source port number)
openflow_flowmod.match.tp_dst  (Destination port number)

When adding flow entries to your switch tables, you should specify as many matching attributes as possible. That is, for TCP and UDP, you should use all five of the attributes above. For ICMP (ping messages), just use the first three, since the meaning of port numbers for ICMP isn't defined.

Note: You also need to set the actions to take on the flowmod message and send it to the switch, just as you do for packet_out commands.

Part 2: Firewall functionality.

After we're able to route packets, we'll add some basic firewall features to enforce a rudimentary security policy. Like the hosts file you're given at controller, start-up, a static set of firewall rules will be passed in under the name firewall. The rules will be of the form:

SwitchDPID action match [arguments]

The switchDPID tells you the identifier of the switch that should enforce this rule.

action is one of:
BLOCK, REDIRECT, or MARK

match is one or more packet header values that must be present to be considered
a match.  Options here are srcip, dstip, protocol, and tos.

[arguments] depends on the action:
For BLOCK, there will be no arguments.
For REDIRECT, the arguments will be: TO [ip address]
For MARK, the arguments will be: TOS [integer identifier]
(TOS stands for "type of service" the header field we'll use for marking.)

When a switch receives an IP packet, you should subject the packet to that switch's rules prior to forwarding it.

A few rule examples:

At the specified switch, block all traffic coming from host 10.1.1.1:
00-00-02-00-00-00-00-00 BLOCK srcip=10.1.1.1

At the specified switch, block all traffic coming from host 10.1.2.2 if the
packet's TOS is marked with 32 and it's destined for 10.1.3.1:
00-00-03-00-00-00-00-00 BLOCK srcip=10.1.2.2 dstip=10.1.3.1 tos=32

At the specified switch, redirect traffic destined for 10.1.2.1 and instead
send it to 10.1.2.2:
00-00-01-00-00-00-00-00 REDIRECT dstip=10.1.2.1 TO 10.1.2.2

At the specified switch, mark the TOS field of all packets sent by 10.1.1.1
with value 40:
00-00-04-00-00-00-00-00 MARK srcip=10.1.1.1 TOS 40

You may assume that multiple rules for the same switch will not overlap or conflict with one another. For simplicity, our firewall only reads and modifies IP packets.

Note that the number of fields in the match portion of the rule varies (there will always be at least one). In order to be considered a match, all of the defined fields must be equal to the specified values.

Just like simple routing, we want to install flow table entries for matching flows so that the switches handle most of the packets without help from the controller. This means that for matching packets, you'll need to set your list of OpenFlow output actions differently. Note that actions is a list and that you can set multiple actions. If you set multiple actions, they will be executed in the order that you specify them. If one of your actions is to send the packet out of a switch port, that needs to be last!

The list of actions you can take are documented in the OpenFlow actions section of the Pox documentation. You should keep the following things in mind as you implement the three actions:

• To block/drop packets at the switch, simply omit the output action. In other words, send the switch a flowmod where the action is to do nothing, and the packets won't be forwarded. When you choose to block a flow, please print a simple message that includes information about the blocked flow so that when I'm grading, I can tell that blocking was intentional (as opposed to a routing bug).


• When marking, the TOS field is actually a six-bit subset of an 8-bit field. This means that the lower two bits will be ignored (and if you set them, you'll get errors and marking WILL NOT work). Make sure any mark values that you use are multiples of 4 to avoid setting the two least significant bits. There is an OpenFlow output action for changing the TOS value of a packet.


• To redirect a packet, you'll need to change both its destination IP address (in the IP header) and the destination MAC address (in the Ethernet header). This should be easy to do, since you already have a way of mapping IP addresses to their corresponding MAC addresses. Just make sure you use two OpenFlow actions: one to rewrite the IP destination, and another to rewrite the MAC destination address.

Tips

• Your code will be significantly easier to reason about if you divide it into multiple helper functions with a good top-down desigh. A large portion of the code you write will live within the _handle_PacketIn method. For your sake and mine, please let _handle_PacketIn call into functions/methods rather than making it one giant mess!


• The format of the hosts file (basichosts/k4hosts) is that of strings. When you receive a packet via a packet_in event, that packet is in a binary format like it would be sent across the wire on a network. You can't compare the two directly. For example, if you look at the hwsrc of an ARP request, you'll see that it's an object of type EthAddr, and if you look at the protosrc, you'll see that it's of type IPAddr. If you want to compare these things against stuff from the hosts file (for instance, if you're doing a lookup in a table that maps IP->MAC addresses), they need to be in the same format (either both binary, or both strings). It's very easy to convert from string to the binary form: call the constructor for those classes with a string argument. E.g., EthAddr("00:01:00:11:00:00") or IPAddr("10.0.0.1"). Likewise, when setting fields in the ARP response, they should be in the format of these objects.


• Similarly, you can convert back and forth between the string representation of a switch's unique identifier (DPID) and a string using the poxutil.dpid_to_str and poxutil.str_to_dpid functions.


• Pox also defines all of the packet formats and objects that you'll need. The files are located in pox/lib/packet/[protocol.py], in case you want to take a look at the fields that are available. This is useful to find out the name of the field you want to set. If you have a packet of type ipv4 and you want to access the ICMP/TCP/UDP fields that it contains, use the packet's .next attribute.


• If your controller is accidentally sending badly formed packets, in addition to printing their contents, you can use the wireshark utility from one of the hosts to dissect the packet for easy readability.

Submitting

Note: You'll need to copy lab2.py from the VM to [your home directory]/cs91/labs/02/ before running handin.

Once you are satisfied with your solution, you can hand it in using the handin91 command from a terminal on any of the CS lab machines.

Only one of you or your partner should run handin91 to submit your joint solutions. If you accidentally both run it, send me email right away letting me know which of the two solutions I should keep and which I should discard.

You may run handin91 as many times as you like, and only the most recent submission will be recorded. This is useful if you realize, after handing in some programs, that you'd like to make a few more changes to them.