Overview

Lecture location: Hudson 201
Lecture time: MW, 3:05PM - 4:20PM

Instructor: Dr. Tyler Bletsch
Email: Tyler.Bletsch AT duke.edu
Office Hours: Mon/Wed, 11AM - 12PM in Hudson 106, or by appointment

Teaching Assistant: David Landry
TA Office Hours: By appointment

Links:

• Piazza forum - get help and discuss course material
• Sakai - submit assignments and see grades
• Feedback form - send anonymous feedback to the instructor
• Rules for access to FitzWest Datacenter - must sign and return to instructor to gain datacenter access

Schedule

#	Date	Lecture	Homework due (11:55:00pm)	Project due (11:55:00pm)
1	Mon 8/28	Course introduction and policies
2	Wed 8/30	Overview of storage systems and project discussion
3	Mon 9/4	Hard disks, SSDs, and the I/O subsystem
4	Wed 9/6	Hard disks, SSDs, and the I/O subsystem	(Fri 9/8) Homework 1
5	Mon 9/11	Hardware failure in storage devices
6	Wed 9/13	RAID		(Fri 9/15) Project proposal due, meeting scheduled
7	Mon 9/18	Network-Attached Storage (NAS)		(9/18-9/22) Project proposal meetings
8	Wed 9/20	Storage Area Network (SAN)	Homework 2
9	Mon 9/25	Filesystems
10	Wed 9/27	Filesystems		(Fri 9/29) Project outlines due, meetings scheduled
11	Mon 10/2	Filesystems		(10/2-10/6) Project outline meetings
12	Wed 10/4	Storage efficiency
	Mon 10/9	Fall break
13	Wed 10/11	Business continuity: High availability	Homework 3
14	Mon 10/16	Business continuity: High availability
15	Wed 10/18	Business continuity: Disaster recovery
16	Mon 10/23	Business continuity: Disaster recovery
17	Wed 10/25	Project milestone presentations (!) (Wed 10/25) NetApp field trip		(Wed 10/25) Project milestone presentations
18	Mon 10/30	Virtual environments
19	Wed 11/1	.~*CLOUD*~.
20	Mon 11/6	.~*CLOUD*~.
21	Wed 11/8	Security
22	Mon 11/13	Data forensics and recovery
23	Wed 11/15	Next-gen storage technologies
24	Mon 11/20	Workload profiling and sizing (Time permitting)		(Mon 11/20) Project demos scheduled
	Wed 11/22	Thanksgiving break
25	Mon 11/27	Project final presentations (!)	★★ Updated 11/10! ★★ Homework 4 (mystery_app, disk-images.tgz)	(11/27-12/8) Project demo meetings (All project materials to be submitted by 12/8)
26	Wed 11/29	Project final presentations (!)
	Wed 12/13	Exam: 9am-12pm

Syllabus & policies

Course synopsis

A chance to study the design and deployment of massive storage systems of the sort used in large enterprises (banks, major IT departments, service providers, etc.). Includes coverage of hard disk and flash design, RAID, SAN and NAS topologies, filesystem design, data center architectures for high availability, data deduplication, business continuity, power aware storage, and the economics of data storage with respect to cloud computing.

Assignments include hands-on lab work with physical servers, some pen-and-paper problems, and semester-long programming project.

Pre-requisites for grad students: ECE 650 (Systems Programming and Engineering) or instructor consent.

Pre-requisites for undergrad students: Computer Science 310/ECE 353 (Operating Systems). Will also need basic networking knowledge (IP addressing, that network switches exist, layer 2 vs layer 3). This can be provided by ECE/COMPSCI 356 (Network Architecture), personal experience, or self-education in parallel with the course.

If you feel you have an OS and networking background but are missing the above pre-reqs, just contact me.

Grading breakdown

This course will require a semester-long project, homework assignments, and a final exam. Grading breakdown:

Category	%
Project proposal	2%
Project outline	3%
Project milestone presentation	5%
Project final presentation	15%
Project demo	20%
Homework	45%
Final exam	10%

Homework

You are expected to complete the homework individually unless otherwise stated. However, you may discuss topics covered in the class.

Late homework submissions incur penalties as follows:

• Submission is 0-24 hours late: total score is multiplied by 0.9
• Submission is 24-48 hours late: total score is multiplied by 0.8
• Submission is more than 48 hours late: total score is multiplied by the Planck constant (as measured in J·s)

NOTE: If you feel in advance that you may need an extension, contact the instructor. We can work with you if you see a scheduling problem coming, but extensions cannot be granted at or near the due date!

Your homework grade will be based on what you submit to Sakai and when you submit it.

Student servers

To support experimentation on real hardware, several storage servers have been procured. Students will split into groups of ~3 and each will be assigned a server. Homeworks will guide students through physically examining, racking, installing, configuring, and using the servers in realistic scenarios.

Some of the hardware is a little dated, but it will exhibit all the usual performance trends, and it has the drives to experiment with RAID topologies, hybrid HDD+SSD storage, filesystem performance, and more. Budget does exist for upgrades if needed (e.g., adding a modern HDD for a comparitive performance study).

The servers will start out in Hudson 01A, but after setting them up, students will install these servers into a standard four-post rack. For this purpose, rack space has been set aside in the "FitzWest" data center in the basement of CIEMAS; students will be granted badge access to this space for this purpose.

NOTE: FitzWest is a real production data center for Duke. Students must exercise caution when working in this space, taking care not to disturb operations of other systems.

In order to guide students through the early phases of server configuration, a few out-of-class lab sessions will be scheduled at a mutually agreeable time. Once servers are configured and deployed properly, all subsequent operations should be able to be conducted over the internet. However, if a physical malfunction occurs (such as drive failure or accidentally trashing the installed OS), students may need to do in-place maintenence of their server within its FitzWest rack.

Any major hardware failures should be reported to the course instructor.

Course reference server

To provide a baseline example of a deployed system, a "reference server" has been configured and installed in FitzWest. The server is a Dell PowerEdge 2950, a 2U rackmount storage system.

Stats:

• Processor: Quad Core Xeon Processor E5310 2x4MB Cache, 1.60GHz, 1066MHz FSB
• Memory: 2GB 667MHz (4X512MB), Single Ranked DIMMs
• Operating system: Ubuntu Linux 16.04 LTS x64
• Storage controller: PERC 5/i, x6 SAS RAID Controller Card
• Storage bays: 1x6 Backplane for 3.5-inch SAS/SATA Hard Drives
• Networking: 2x 1GbE ethernet. One uplink connected at present.
• Drives:
  • [3x] Western Digital 250GB 7200rpm SATA 3Gbps 3.5-in HDD (circa 2007)
  • [1x] Samsung 850 EVO SSD, SATA, 250GB (new)
  • [1x] Zheino SSD, SATA, 30GB (the cheapest SSD on Amazon today)
  • [1x] Sandisk USB thumb drive, 30GB (contains the OS, not for testing!)
• Features: Redundant Power Supply, out-of-band BMC management via IPMI

This server is up at storemaster.egr.duke.edu from campus or via VPN.

Grade appeals

All regrade requests must be in writing. Email the TA with your questions. After speaking with the TA, if you still have concerns, contact the instructor.

All regrade requests must be submitted to the instructor no later than 1 week after the assignment was returned to you.

Academic integrity

I take academic integrity extremely seriously. Academic misconduct will not be tolerated, and all suspected violations of the Duke Honor Code will be referred to the Office of Student Conduct (for undergraduates) or the departmental Director of Graduate Studies (for graduate students). A student found responsible for academic dishonesty faces formal disciplinary action, which may include suspension. A student twice suspended automatically faces a minimum 5-year separation from Duke University.

In addition to the measures taken by the university, the affected assignment(s) will receive zero credit, or possibly -100% in egregious cases.

If you are considering this course of action, please see me instead, and we can work something out! I want every student in my course to be successful.