Overview

Lecture location: Hudson 216
Lecture time: Mon/Wed, 1:25pm - 2:40pm

Instructor: Dr. Tyler Bletsch
Email: Tyler.Bletsch AT duke.edu
Office Hours: After class or by appointment (feel free to email me)

Teaching Assistant: Tom Chang (jen-hung.chang AT duke.edu)

Links:

• Ed forum - get help and discuss course material
• Canvas - submit code and see grades
• GradeScope - submit homework/lab assignments and get feedback
• Feedback form - send anonymous feedback to the instructor
• Rules for access to FitzWest Datacenter - must agree electronically to gain datacenter access

Schedule

#	Date	Lecture	Assignment due (11:59:00pm)	Project due (11:59:00pm)
1	Wed 1/7	Course introduction and storage system overview
2	Mon 1/12	Storage system overview, continued
3	Wed 1/14	Hard disks, SSDs, and the I/O subsystem	(Wed 1/14) FitzWest Datacenter Rules Agreement Lab 0 session #1 Where: TBA When: TBA
	Mon 1/19	MLK day
4	Wed 1/21	Hard disks, SSDs, and the I/O subsystem	Lab 0 session #2 Where: TBA When: TBA
5	Mon 1/26	Hard disks, SSDs, and the I/O subsystem
6	Wed 1/28	Hardware failure in storage devices	(Wed 1/28) Lab 0
7	Mon 2/2	RAID
8	Wed 2/4	RAID	(Wed 2/4) Program: FUSE
9	Mon 2/9	Network-Attached Storage (NAS)
10	Wed 2/11	Storage Area Network (SAN)	(Wed 2/11) Homework 1, Lab 1
11	Mon 2/16	Project planning discussion; Filesystems
12	Wed 2/18	Filesystems	(Wed 2/18) Program: BUSE (RAID 0 only)	(Fri 2/20) Project proposal draft due; meeting scheduled
13	Mon 2/23	Storage efficiency		(2/23 - 2/27) Project draft meetings
14	Wed 2/25	Business continuity: High availability	(Wed 2/25) Program: BUSE (everything)
15	Mon 3/2	Business continuity: High availability	(Mon 3/2) Homework 2
16	Wed 3/4	Business continuity: Disaster recovery		(Fri 3/6) Project proposal due
	Mon 3/9	SPRING BREAK
	Wed 3/11	SPRING BREAK
17	Mon 3/16	Business continuity: Disaster recovery	(Tue 3/17) Lab 2	(wow, what a nice week to get  a huge jump on the project!)
18	Wed 3/18	Midterm review; Virtual environments
19	Mon 3/23	Midterm exam
20	Wed 3/25	Project workday and milestone demo		(Wed 3/25, 8:00AM) Project milestone 1 due
21	Mon 3/30	Cloud	(Mon 3/30) Lab 3
22	Wed 4/1	Workload profiling and sizing	(Wed 4/1) Homework 3
23	Mon 4/6	Security
24	Wed 4/8	Project workday and milestone demo		(Wed 4/8, 8:00AM) Project milestone 2 due
25	Mon 4/13	Data forensics and recovery		(Mon 4/13) Project demos scheduled
26	Wed 4/15	Next-gen storage technologies	(Wed 4/15) Homework 4, Lab 4
27	Mon 4/20	Project final presentations		(Mon 4/20) Project materials due (Tue 4/21 - Fri 4/24) Project demo meetings
	Tue 4/28	Final exam, 2:00pm

Syllabus & policies

Course synopsis

A chance to study the design and deployment of massive storage systems of the sort used in large enterprises (banks, major IT departments, service providers, etc.). Includes coverage of hard disk and flash design, RAID, SAN and NAS topologies, filesystem design, data center architectures for high availability, data deduplication, business continuity, power aware storage, and the economics of data storage with respect to cloud computing.

Assignments include hands-on lab work with physical servers, some pen-and-paper problems, and semester-long programming project.

Pre-requisites for grad students: ECE 650 (Systems Programming and Engineering) or instructor consent.

Pre-requisites for undergrad students: Computer Science 310/ECE 353 (Operating Systems). Will also need basic networking knowledge (IP addressing, that network switches exist, layer 2 vs layer 3). This can be provided by ECE/COMPSCI 356 (Network Architecture), personal experience, or self-education in parallel with the course.

If you feel you have an OS and networking background but are missing the above pre-reqs, just contact me.

Grading breakdown

This course will require a semester-long project, homework assignments, and a midterm+final exam. Grading breakdown:

Category	%
Project proposal draft	2%
Project proposal	3%
Project milestone 1	5%
Project milestone 2	5%
Project final report	5%
Project final presentation	5%
Project final demo	15%
Homeworks/programs/labs	40%
Midterm exam	10%
Final exam	10%

Homework and Labs

There are two kinds of regular assigments, homeworks and labs:

• Homeworks are individual and include written exercises to grow your understanding of the theory.
• Labs are group work that include hands-on tasks to do with real storage equipment.

In either case, you are free to discuss concepts covered in the class with others (other groups for lab work and other people for individual homework), but should not share answers or concrete steps oustide the bounds of academic integrity.

Late homework/lab submissions incur penalties as follows:

• Submission is 0-24 hours late: total score is multiplied by 0.9
• Submission is 24-48 hours late: total score is multiplied by 0.8
• Submission is more than 48 hours late: total score is multiplied by the Planck constant (as measured in J·s)

NOTE: If you feel in advance that you may need an extension, contact the instructor. We can work with you if you see a scheduling problem coming, but extensions cannot be granted at or near the due date!

Your homework/lab grade will be based on what you submit to Sakai and when you submit it.

Student servers

To support experimentation on real hardware, several storage servers have been procured. Students will split into groups of ~3 and each will be assigned a server. Homeworks will guide students through physically examining, racking, installing, configuring, and using the servers in realistic scenarios.

Some of the hardware is a little dated, but it will exhibit all the usual performance trends, and it has the drives to experiment with RAID topologies, hybrid HDD+SSD storage, filesystem performance, and more. Budget does exist for upgrades if needed (e.g., adding a modern HDD for a comparitive performance study).

The servers will start out in Hudson 06, but after setting them up, students will install these servers into a standard four-post rack. For this purpose, rack space has been set aside in the "FitzWest" data center in the basement of CIEMAS; students will be granted badge access to this space for this purpose.

NOTE: FitzWest is a real production data center for Duke. Students must exercise caution when working in this space, taking care not to disturb operations of other systems.

In order to guide students through the early phases of server configuration, a few out-of-class lab sessions will be scheduled at a mutually agreeable time. Once servers are configured and deployed properly, all subsequent operations should be able to be conducted over the internet. However, if a physical malfunction occurs (such as drive failure or accidentally trashing the installed OS), students may need to do in-place maintenence of their server within its FitzWest rack.

Any major hardware failures should be reported to the course instructor.

Grade appeals

All regrade requests must be in writing. Email the TA with your questions. After speaking with the TA, if you still have concerns, contact the instructor.

All regrade requests must be submitted to the instructor no later than 1 week after the assignment was returned to you.

AI Policy

The use of AI to generate anything you turn in is not allowed. However, you can use such systems to help study or diagnose issues. Here is a very clear rule to follow:

The number of LLM-generated bytes in anything you submit should be exactly zero
(except where explicitly allowed by the instructor).

AI Usage Appendix: Further, all assignments must include an AI Usage Appendix at the end. Content:

• You must summarize the kinds of prompts used and how the output helped you (you don’t have to give every prompt/response), if any.
• If you didn’t touch AI at all, then you can just say that.
• Either way, conclude by asserting that zero bytes of LLM output appear in your submission (except where explicitly permitted by the instructor, if applicable).

Exceptions: Limited use may be allowed with prior instructor approval for aspects of the project that are not core to this course. For example: If building a filesystem that indexes PDF files by metadata, you could request to use AI to generate code for PDF metadata extraction, since that’s not core to the course. You must get my written approval before doing so. Your project report would include this use in detail in the AI Usage Appendix.

Here's an example AI Usage Appendix where it was briefly consulted to fix a technical issue:

Here's an example AI Usage Appendix where no LLM was used at all: Enforcement: I reserve the right to ask you the reasoning behind anything you submit; this may occur without prior notice. If you can’t explain your own reasoning at all, I reserve the right to file an academic integrity complaint on that basis.

Academic integrity

I take academic integrity extremely seriously. Academic misconduct will not be tolerated, and all suspected violations of the Duke Honor Code will be referred to the Office of Student Conduct (for undergraduates) or the departmental Director of Graduate Studies (for graduate students). A student found responsible for academic dishonesty faces formal disciplinary action, which may include suspension. A student twice suspended automatically faces a minimum 5-year separation from Duke University.

In addition to the measures taken by the university, the affected assignment(s) will receive zero credit, or possibly -100% in egregious cases.

If you are considering this course of action, please see me instead, and we can work something out! I want every student in my course to be successful.