The invention of RAID probably ruined this guy's life.
Overview
Lecture location: Hudson 222
Lecture time: TuTh, 11:45AM - 1:00PM
Instructor: Dr. Tyler Bletsch
Email: Tyler.Bletsch AT duke.edu
Office Hours: Mon/Wed, 1:30pm-3pm in Hudson 106
Teaching Assistant: Andrew Stevens (Andrew.J.Stevens AT duke.edu)
TA Office Hours: See Piazza post
Links:
- Piazza forum - get help and discuss course material
- Sakai - submit assignments and see grades
- Suggestion box - send anonymous feedback to the instructor
Schedule
| # | Date | Lecture | Homework due (11:55:00pm) | Project due (11:55:00pm) |
|---|---|---|---|---|
| 1 | Tue 8/30 | Course introduction and policies | ||
| 2 | Thu 9/1 | Overview of storage systems and project discussion | ||
| 3 | Tue 9/6 | Hard disks, SSDs, and the I/O subsystem | ||
| 4 | Thu 9/8 | Hard disks, SSDs, and the I/O subsystem | ||
| 5 | Tue 9/13 | Hardware failure in storage devices | ||
| 6 | Thu 9/15 | RAID | (Fri 9/16) Project proposals submitted+scheduled | |
| 7 | Tue 9/20 | Network-Attached Storage (NAS) | (9/19-9/23) Project proposal meetings | |
| 8 | Thu 9/22 | Storage Area Network (SAN) | ||
| 9 | Tue 9/27 | Filesystems | Homework 1 | |
| 10 | Thu 9/29 | Filesystems | (Fri 9/30) Project outlines submitted+scheduled | |
| 11 | Tue 10/4 | Filesystems | (10/3-10/7) Project outline meetings | |
| 12 | Thu 10/6 | Storage efficiency | ||
| Tue 10/11 | Fall break | |||
| 13 | Thu 10/13 | Business continuity: High availability | ||
| 14 | Tue 10/18 | Business continuity: High availability | ||
| 15 | Thu 10/20 | Business continuity: Disaster recovery | ||
| 16 | Tue 10/25 | Business continuity: Disaster recovery | ||
| 17 | Thu 10/27 | Project milestone presentations
(5pm) NetApp field trip | (Thu 10/27) Project milestone presentations | |
| 18 | Tue 11/1 | Virtual environments | Homework 2 | |
| 19 | Thu 11/3 | .~*CLOUD*~. | ||
| 20 | Tue 11/8 | .~*CLOUD*~. | ||
| 21 | Thu 11/10 | Security | ||
| 22 | Tue 11/15 | Security | ||
| 23 | Thu 11/17 | Workload profiling and sizing
Paper: Gulati et al.
| ||
| 24 | Tue 11/22 | Next-gen storage technologies | (Tue 11/22) Project demos scheduled. | |
| Thu 11/24 | Thanksgiving break | |||
| 25 | Tue 11/29 | Project final presentations | Homework 3 | (11/30-12/9) Project final demo meetings |
| 26 | Thu 12/1 | Project final presentations | ||
| Thu 12/15 | Exam: 2pm-5pm Study guide |
Field trip to NetApp
We'll be doing a field trip to NetApp. There, we'll hear about the storage controller software development as well as architecting complete customer environments and get a tour of their giant datacenter.
- Date: Thursday, 27 October 2016
- Time: 5pm-7:30pm
- Address: 7301 Kit Creek Rd, Durham, NC 27709
- Where to go: Upon arrival, please follow signs to Building 1 and enter through the main lobby. Parking will be available directly in-front of the building.
Syllabus & policies
Course synopsis
A chance to study the design and deployment of massive storage systems of the sort used in large enterprises (banks, major IT departments, service providers, etc.). Includes coverage of hard disk and flash design, RAID, SAN and NAS topologies, filesystem design, data center architectures for high availability, data deduplication, business continuity, power aware storage, and the economics of data storage with respect to cloud computing.Includes a few homeworks and a semester-long programming project.
Pre-requisites for grad students: ECE 650 (Systems Programming and Engineering) or instructor consent.
Pre-requisites for undergrad students: COMPSCI 310 (OS) and ECE/COMPSCI 356 (Network Architecture) or instructor consent. The 356 can be taken concurrently.
If you feel you have an OS and networking background but are missing the above pre-reqs, just contact me.
Grading breakdown
This course will require a semester-long project, homework assignments, and a final exam. Grading breakdown:
| Category | % |
|---|---|
| Project proposal | 5% |
| Project outline | 5% |
| Project milestone presentation | 5% |
| Project final presentation | 15% |
| Project demo | 20% |
| Homework | 30% |
| Final exam | 20% |
Homework
You are expected to complete the homework individually unless otherwise stated. However, you may discuss topics covered in the class.
Late homework submissions incur penalties as follows:
- Submission is 0-24 hours late: total score is multiplied by 0.9
- Submission is 24-48 hours late: total score is multiplied by 0.8
- Submission is more than 48 hours late: total score is multiplied by the Planck constant (as measured in J·s)
NOTE: If you feel in advance that you may need an extension, contact the instructor. We can work with you if you see a scheduling problem coming, but extensions cannot be granted at or near the due date!
Your homework grade will be based on what you submit to Sakai and when you submit it.
Course storage server
The server under construction on my desk.
A 2.5" SSD mounted in the 3.5" drive tray
with a 3D-printed adapter.
Stats:
- Processor: Quad Core Xeon Processor E5310 2x4MB Cache, 1.60GHz, 1066MHz FSB
- Memory: 2GB 667MHz (4X512MB), Single Ranked DIMMs
- Operating system: Ubuntu Linux 16.04 LTS x64
- Storage controller: PERC 5/i, x6 SAS RAID Controller Card
- Storage bays: 1x6 Backplane for 3.5-inch SAS/SATA Hard Drives
- Networking: 2x 1GbE ethernet. One uplink connected at present.
- Drives:
- [3x] Western Digital 250GB 7200rpm SATA 3Gbps 3.5-in HDD (circa 2007)
- [1x] Samsung 850 EVO SSD, SATA, 250GB (new)
- [1x] Zheino SSD, SATA, 30GB (the cheapest SSD on Amazon today)
- [1x] Sandisk USB thumb drive, 30GB (
contains the OS, not for testing! )
- Features: Redundant Power Supply, out-of-band BMC management via IPMI
Access it via SSH at storemaster.egr.duke.edu from campus or via VPN. User accounts created upon request, including root access.
Grade appeals
All regrade requests must be in writing. Email the TA with your questions. After speaking with the TA, if you still have concerns, contact the instructor.All regrade requests must be submitted to the instructor no later than 1 week after the assignment was returned to you.
Academic integrity
I take academic integrity extremely seriously. Academic misconduct will not be tolerated, and all suspected violations of the Duke Honor Code will be referred to the Office of Student Conduct (for undergraduates) or the departmental Director of Graduate Studies (for graduate students). A student found responsible for academic dishonesty faces formal disciplinary action, which may include suspension. A student twice suspended automatically faces a minimum 5-year separation from Duke University.In addition to the measures taken by the university, the affected assignment(s) will receive zero credit, or possibly -100% in egregious cases.
If you are considering this course of action, please see me instead, and we can work something out! I want every student in my course to be successful.