ECE 356 Computer Network Architecture: Lab 2 Simple Router
Start Soon
This assignment might take you longer time than you think to finish, especially when you are not familar with Linux ANSI C programming, Wireshark or TCP/IP. So START SOON!
Introduction
In this lab assignment you will be writing a simple router with a static routing table. Your router will receive raw Ethernet frames. It will process the packets just like a real router, then forward them to the correct outgoing interface. We'll make sure you receive the Ethernet frames; your job is to create the forwarding logic so packets go to the correct interface.
Your router will route real packets from a emulated host (client) to two emulated application servers (http server 1/2) sitting behind your router. The application servers are each running an HTTP server. When you have finished the forwarding path of your router, you should be able to access these servers using regular client software. In addition, you should be able to ping and traceroute to and through a functioning Internet router. A sample routing topology is shown below:
If the router is functioning correctly, all of the following operations should work:
- Pinging from the client to any of the router's interfaces (192.168.2.1, 172.64.3.1, 10.0.1.1).
- Tracerouting from the client to any of the router's interfaces
- Pinging from the client to any of the app servers (192.168.2.2, 172.64.3.10)
- Tracerouting from the client to any of the app servers
- Downloading a file using HTTP from one of the app servers
Collaboration Policy
You should finish this assignment in a group of up to 2 persons.
Mininet & POX
This assignment runs on top of Mininet & POX. Mininet allows you to emulate a topology on a single machine. POX is an OpenFlow controller. It can control routers in Mininet. In this assignment, the router redirects every packets to POX controller, then POX controller redirects them to your code. You don't have to know how they work to complete this assignment.
Get Started
Environment Setup
All these frameworks run on Linux operating system. Configure them manually might requires a lot of time. They have been configured in the virtual machine image.
/home/ubuntu/Projects/cs144_lab3/
The pathname contains "cs144_lab3" because this assignment is the third assignment in Stanford CS144 course. I ported their assignment directly. You should modify the files under directory /home/ubuntu/Projects/cs144_lab3/router/ to complete this assignment.
Test Connectivity of Your Emulated Topology
Start Mininet emulation by using the following command
cd ~/Projects/cs144_lab3/
./run_mininet.sh
You should be able to see some output like the following:
*** Shutting down stale SimpleHTTPServers
*** Shutting down stale webservers
server1 192.168.2.2
server2 172.64.3.10
client 10.0.1.100
sw0-eth1 192.168.2.1
sw0-eth2 172.64.3.1
sw0-eth3 10.0.1.1
*** Successfully loaded ip settings for hosts
{'server1': '192.168.2.2', 'sw0-eth3': '10.0.1.1', 'sw0-eth1': '192.168.2.1', 'sw0-eth2': '172.64.3.1', 'client': '10.0.1.100', 'server2': '172.64.3.10'}
*** Creating network
*** Creating network
*** Adding controller
*** Adding hosts:
client server1 server2
*** Adding switches:
sw0
*** Adding links:
(client, sw0) (server1, sw0) (server2, sw0)
*** Configuring hosts
client server1 server2
*** Starting controller
*** Starting 1 switches
sw0
*** setting default gateway of host server1
server1 192.168.2.1
*** setting default gateway of host server2
server2 172.64.3.1
*** setting default gateway of host client
client 10.0.1.1
*** Starting SimpleHTTPServer on host server1
*** Starting SimpleHTTPServer on host server2
*** Starting CLI:
mininet>
Keep this terminal open, as you will need the mininet command line for debugging. Now, use another terminal to continue the next step. To open a new terminal in the same directory of current terminal, you can right click current terminal and press "Open Terminal" or "Open Tab".
Mininet requires a controller, which we implemented in POX. To run the controller, use the following command to run the controller:
cd ~/Projects/cs144_lab3/
./run_pox.sh
You should be able to see some output like the following:
POX 0.2.0 (carp) / Copyright 2011-2013 James McCauley, et al.
INFO:.home.ubuntu.projects.cs144_lab3.pox_module.cs144.srhandler:created server
INFO:core:POX 0.2.0 (carp) is up.
INFO:openflow.of_01:[16-a5-39-7e-79-41 1] connected
Now you are ready to test out the connectivity of the environment setup. To do so, run the binary file of the solution “sr_solution”
cd ~/Projects/cs144_lab3/
./sr_solution
You should be able to see some output like the following:
Using VNS sr stub code revised 2009-10-14 (rev 0.20)
Loading routing table from server, clear local routing table.
Loading routing table
---------------------------------------------
Destination Gateway Mask Iface
10.0.1.100 10.0.1.100 255.255.255.255 eth3
192.168.2.2 192.168.2.2 255.255.255.255 eth1
172.64.3.10 172.64.3.10 255.255.255.255 eth2
---------------------------------------------
Client ubuntu connecting to Server localhost:8888
Requesting topology 0
successfully authenticated as ubuntu
Loading routing table from server, clear local routing table.
Loading routing table
---------------------------------------------
Destination Gateway Mask Iface
10.0.1.100 10.0.1.100 255.255.255.255 eth3
192.168.2.2 192.168.2.2 255.255.255.255 eth1
172.64.3.10 172.64.3.10 255.255.255.255 eth2
---------------------------------------------
Router interfaces:
eth3 HWaddrf2:87:91:7d:2e:7a
inet addr 10.0.1.1
eth2 HWaddr46:d5:1e:89:53:0d
inet addr 172.64.3.1
eth1 HWaddrbe:3c:a3:e6:1a:43
inet addr 192.168.2.1
<-- Ready to process packets -->
In this particular setup, 192.168.2.2 is the IP for server1, and 172.64.3.10 is the IP for server2. You can find the IP addresses in a file called IP_CONFIG.
Now, back to the terminal where Mininet is running. To issue an command on the emulated host, type the host name followed by the command in the Mininet console. For example, the following command issues 3 pings from the client to the server1.
mininet> client ping -c 3 192.168.2.2
You should be able to see the following output.
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_req=1 ttl=63 time=66.9 ms
64 bytes from 192.168.2.2: icmp_req=2 ttl=63 time=49.9 ms
64 bytes from 192.168.2.2: icmp_req=3 ttl=63 time=68.8 ms
You can also use traceroute to see the route between client to server1.
mininet> client traceroute -n 192.168.2.2
You should be able to see the following output.
traceroute to 192.168.2.2 (192.168.2.2), 30 hops max, 60 byte packets
1 10.0.1.1 146.069 ms 143.739 ms 143.523 ms
2 192.168.2.2 226.260 ms 226.070 ms 225.868 ms
Finally, to test the web server is properly working at the server1 and server2, issue an HTTP request by using wget or curl.
mininet> client wget http://192.168.2.2
You should be able to see the following output.
--2012-12-17 06:52:23-- http://192.168.2.2/
Connecting to 192.168.2.2:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 161 [text/html]
Saving to: `index.html'
0K 100% 17.2M=0s
2012-12-17 06:52:24 (17.2 MB/s) - `index.html' saved [161/161]
If you stop the './sr_solution', you will find the ping/traceroute/wget won't work anymore. In this assignment, you will replicate the functionality of sr_solution. To help you get started, we provide some starter code described in the following section..
We will use these test cases to grade your code.
Starter Code
You should now have all the pieces needed to build and run the router:
The starter source code
You can build and run the starter code as follows:
cd ~/Projects/cs144_lab3/router/
make
./sr
General Forwarding Logic
To get you started, an outline of the forwarding logic for a router follows, although it does not contain all the details. There are two main parts to this assignment: Handling ARP and IP forwarding
IP Forwarding
Given a raw Ethernet frame, if the frame contains an IP packet that is not destined towards one of our interfaces:
- Sanity-check the packet (meets minimum length and has correct checksum).
- Decrement the TTL by 1, and recompute the packet checksum over the modified header.
- Find out which entry in the routing table has the longest prefix match with the destination IP address.
- Check the ARP cache for the next-hop MAC address corresponding to the next-hop IP. If it's there, send it. Otherwise, send an ARP request for the next-hop IP (if one hasn't been sent within the last second), and add the packet to the queue of packets waiting on this ARP request. Obviously, this is a very simplified version of the forwarding process, and the low-level details follow. For example, if an error occurs in any of the above steps, you will have to send an ICMP message back to the sender notifying them of an error. You may also get an ARP request or reply, which has to interact with the ARP cache correctly.
Protocols to Understand
Ethernet
You are given a raw Ethernet frame and have to send raw Ethernet frames. You should understand source and destination MAC addresses and the idea that we forward a packet one hop by changing the destination MAC address of the forwarded packet to the MAC address of the next hop's incoming interface.
Internet Protocol
Before operating on an IP packet, you should verify its checksum and make sure it meets the minimum length of an IP packet. You should understand how to find the longest prefix match of a destination IP address in the routing table described in the "Getting Started" section. If you determine that a datagram should be forwarded, you should correctly decrement the TTL field of the header and recompute the checksum over the changed header before forwarding it to the next hop.
Internet Control Message Protocol
ICMP is a simple protocol that can send control information to a host. In this assignment, your router will use ICMP to send messages back to a sending host. You will need to properly generate the following ICMP messages (including the ICMP header checksum) in response to the sending host under the following conditions:
- Echo reply (type 0) Sent in response to an echo request (ping) to one of the router's interfaces. (This is only for echo requests to any of the router's IPs. An echo request sent elsewhere should be forwarded to the next hop address as usual.)
- Destination net unreachable (type 3, code 0) ** Sent if there is a non-existent route to the destination IP (no matching entry in routing table when forwarding an IP packet).
- Destination host unreachable (type 3, code 1) ** Sent if five ARP requests were sent to the next-hop IP without a response.
- Port unreachable (type 3, code 3) ** Sent if an IP packet containing a UDP or TCP payload is sent to one of the router's interfaces. This is needed for traceroute to work.
- Time exceeded (type 11, code 0) ** Sent if an IP packet is discarded during processing because the TTL field is 0. This is also needed for traceroute to work. The source address of an ICMP message can be the source address of any of the incoming interfaces, as specified in RFC 792. As mentioned above, the only incoming ICMP message destined towards the router's IPs that you have to explicitly process are ICMP echo requests. You may want to create additional structs for ICMP messages for convenience, but make sure to use the packed attribute so that the compiler doesn't try to align the fields in the struct to word boundaries:
Address Resolution Protocol
ARP is needed to determine the next-hop MAC address that corresponds to the next-hop IP address stored in the routing table. Without the ability to generate an ARP request and process ARP replies, your router would not be able to fill out the destination MAC address field of the raw Ethernet frame you are sending over the outgoing interface. Analogously, without the ability to process ARP requests and generate ARP replies, no other router could send your router Ethernet frames. Therefore, your router must generate and process ARP requests and replies. To lessen the number of ARP requests sent out, you are required to cache ARP replies. Cache entries should time out after 15 seconds to minimize staleness. The provided ARP cache class already times the entries out for you. When forwarding a packet to a next-hop IP address, the router should first check the ARP cache for the corresponding MAC address before sending an ARP request. In the case of a cache miss, an ARP request should be sent to a target IP address about once every second until a reply comes in. If the ARP request is sent five times with no reply, an ICMP destination host unreachable is sent back to the source IP as stated above. The provided ARP request queue will help you manage the request queue. In the case of an ARP request, you should only send an ARP reply if the target IP address is one of your router's IP addresses. In the case of an ARP reply, you should only cache the entry if the target IP address is one of your router's IP addresses. Note that ARP requests are sent to the broadcast MAC address (ff-ff-ff-ff-ff-ff). ARP replies are sent directly to the requester's MAC address.
IP Packet Destinations
An incoming IP packet may be destined for one of your router's IP addresses, or it may be destined elsewhere. If it is sent to one of your router's IP addresses, you should take the following actions, consistent with the section on protocols above:
- If the packet is an ICMP echo request and its checksum is valid, send an ICMP echo reply to the sending host.
- If the packet contains a TCP or UDP payload, send an ICMP port unreachable to the sending host. Otherwise, ignore the packet. Packets destined elsewhere should be forwarded using your normal forwarding logic.
Code Overview
Basic Functions
Your router receives a raw Ethernet frame and sends raw Ethernet frames when sending a reply to the sending host or forwarding the frame to the next hop. The basic functions to handle these functions are:
void sr_handlepacket(struct sr_instance* sr, uint8_t * packet, unsigned int len, char* interface)
This method, located in sr_router.c, is called by the router each time a packet is received. The "packet" argument points to the packet buffer which contains the full packet including the ethernet header. The name of the receiving interface is passed into the method as well.
int sr_send_packet(struct sr_instance* sr, uint8_t* buf, unsigned int len, const char* iface)
This method, located in sr_vns_comm.c, will send an arbitrary packet of length, len, to the network out of the interface specified by iface.
You should not free the buffer given to you in sr_handlepacket (this is why the buffer is labeled as being "lent" to you in the comments). You are responsible for doing correct memory management on the buffers that sr_send_packet borrows from you (that is, sr_send_packet will not call free on the buffers that you pass it).
void sr_arpcache_sweepreqs(struct sr_instance *sr)
The assignment requires you to send an ARP request about once a second until a reply comes back or we have sent five requests. This function is defined in sr_arpcache.c and called every second, and you should add code that iterates through the ARP request queue and re-sends any outstanding ARP requests that haven't been sent in the past second. If an ARP request has been sent 5 times with no response, a destination host unreachable should go back to all the sender of packets that were waiting on a reply to this ARP request.
Data Structures
The Router (sr_router.h):
The full context of the router is housed in the struct sr_instance (sr_router.h). sr_instance contains information about topology the router is routing for as well as the routing table and the list of interfaces.
Interfaces (sr_if.c/h):
After connecting, the server will send the client the hardware information for that host. The stub code uses this to create a linked list of interfaces in the router instance at member if_list. Utility methods for handling the interface list can be found at sr_if.c/h.
The Routing Table (sr_rt.c/h):
The routing table in the stub code is read on from a file (default filename "rtable", can be set with command line option -r ) and stored in a linked list of routing entries in the current routing instance (member routing_table).
The ARP Cache and ARP Request Queue (sr_arpcache.c/h):
You will need to add ARP requests and packets waiting on responses to those ARP requests to the ARP request queue. When an ARP response arrives, you will have to remove the ARP request from the queue and place it onto the ARP cache, forwarding any packets that were waiting on that ARP request. Pseudocode for these operations is provided in sr_arpcache.h. The base code already creates a thread that times out ARP cache entries 15 seconds after they are added for you. You must fill out the sr_arpcache_sweepreqs function in sr_arpcache.c that gets called every second to iterate through the ARP request queue and re-send ARP requests if necessary. Psuedocode for this is provided in sr_arpcache.h.
Protocol Headers (sr_protocol.h)
Within the router framework you will be dealing directly with raw Ethernet packets. The stub code itself provides some data structures in sr_protocols.h which you may use to manipulate headers easily. There are a number of resources which describe the protocol headers in detail. Network Sorcery's RFC Sourcebook provides a condensed reference to the packet formats you'll be dealing with:
- Ethernet
- IP
- ICMP
- ARP
Debugging Functions
We have provided you with some basic debugging functions in sr_utils.h, sr_utils.c. Feel free to use them to print out network header information from your packets. Below are some functions you may find useful:
- print_hdrs(uint8_t *buf, uint32_t length) - Prints out all possible headers starting from the Ethernet header in the packet
- print_addr_ip_int(uint32_t ip) - Prints out a formatted IP address from a uint32_t. Make sure you are passing the IP address in the correct byte ordering.
Reference Binary
To help you debug your topologies and understand the required behavior we provide a reference binary and you can find it at ~/Projects/cs144_lab3/sr_solution in your directory:
Required Functionality
- The router must successfully route packets between the Internet and the application servers.
- The router must correctly handle ARP requests and replies.
- The router must correctly handle traceroutes through it (where it is not the end host) and to it (where it is the end host).
- The router must respond correctly to ICMP echo requests.
- The router must handle TCP/UDP packets sent to one of its interfaces. In this case the router should respond with an ICMP port unreachable.
- The router must maintain an ARP cache whose entries are invalidated after a timeout period (timeouts should be on the order of 15 seconds).
- The router must queue all packets waiting for outstanding ARP replies. If a host does not respond to 5 ARP requests, the queued packet is dropped and an ICMP host unreachable message is sent back to the source of the queued packet.
- The router must not needlessly drop packets (for example when waiting for an ARP reply)
- The router must enforce guarantees on timeouts--that is, if an ARP request is not responded to within a fixed period of time, the ICMP host unreachable message is generated even if no more packets arrive at the router. (Note: You can guarantee this by implementing the sr_arpcache_sweepreqs function in sr_arpcache.c correctly.)
Development Tips
Use Eclipse CDT
To accelarate development, I suggest to use an integrated development environment (IDE). Eclipse is a good choice. Note that you are not required to use Eclipse. It's just a recommendation. I installed Eclipse with C++/C Development Toolkit in the virtual machine. Eclipse provides good project management, content assistance and a good graphical wrapper of GDB. The directory of Eclipse is:
/home/ubuntu/Downloads/eclipse/
You can also open Eclipse with the link on Desktop. I have already imported the project into Eclipse. You will see the project once you open Eclipse. Here are some brief steps to use Eclipse:
- Modify some code
- Press "save all" button on top of Eclipse or press "File->Save All".
- Press "build" button (looks like a hammer) or press "Project->Build All"
- Right click in the editor, press "Run As->Local C++/C Application"
- See the result in the "Console" tab on the bottom of Eclipse
Debugger
Eclipse has a good graphical GDB wrapper. I assume you have some basic knowledge of debugger, like "step into", "step over" and "breakpoint". After you have built your project, right click in the editor, press "Debug As->Local C++/C Application". Then you will see the organization of panel changed. Then Eclipse will stop the program at the first line of your main function. You will see a blue arrow on the left of your code:
You can step into a function call by pressing "F5" or step over a function call by pressing "F6". You can insert a breakpoint in a certain line by double left click the left side of the line. You will see a little blue circle on it:
Debugger will stop the program when it encounters a breakpoint.
At any point, you can observe the value of variables on the "Variables" tab on the right top corner:
Somethimes you may want to observe the value of a given expression, like "a + b". You can press "Window->Show View->Expressions" then you will a new "Expressions" tab on the right top corner:
You can insert the content of the expression you want to observe in it.
This assignment deals with a lot of memory and pointer operations. All packets are blocks on the memory. You may want to observe the content of a memroy block. You can press "Window->Show View->Memory", then you will see a new "Memory" tab inserted on the bottom of Eclipse:
You can get the address of pointer with "Variables" or "Expressions" tab. Suppose you want to observe the address "0x123456", press the "add" button on "Memory" tab, type "0x123456" then you will see the content on the right-hand side of "Memory" tab.
Wireshark
Wireshark is a free network packet analyzer. It can be used to troubleshoot many network problems. It capsulates packets and dump their content. I have already installed an old version of Wireshark on the virtual machine.
As mentioned before, the topology of this assignment contains four nodes, client, router, and two servers. You can run Wireshark can capsulate packet in any node. For instance to run Wireshark in client:
- In Mininet terminal, type "xterm client" (not "client xterm")
- In the newly opened xterm, type "wireshark"
- Ignore all the warnings, press "Capture->Interface"
- Press "Start" button on the right side of "client-eth0". Then wireshark will capture and dump packets in/out of client's ethernet interface.
Note that you can press "Ctrl+E" to stop or start capturing packets.
Suppose in a centain point during your development, you try to ping gateway by typing "client ping 10.0.1.1" in the Mininet terminal. It gives such output:
That indicates your simple router cannot handle ICMP packet well. However, many reasons can lead to such problem. It is better to use Wireshark to know what happends in reality. Start the Wireshark following steps above. Suppose you see packtes like this:
You can see that all the packets are ARP requests from client and router gives no reply. Then you know that your code cannot handle ARP requests and send ARP reply correctly. This simple example shows that whenever your code doesn't work, try to use Wireshark to figure out what packet simple router receives and sends. Besides, you can also capture packets send by reference binary and observe the correct content of packets.
How to Submit
Go to the router folder, create a file named "README" with your group members' names, and
make submit
Upload the compressed file on Canvas.
Acknowledgement
We thank the course staff of CS144 at Stanford University for their support. This lab is adapted from one of their assignments. You can visit their Wiki page here.