EECS 582: Advanced Operating Systems (F’16)

 

Announcements

  • 09/29/2016: Change of class schedule in last couple of weeks.
  • 08/24/2016: Paper summary site is online. Create an account ASAP with your umich.edu email address to get reviewer access!
  • 08/02/2016: Welcome to EECS 582!

Administrivia

Course EECS 582, Fall 2016, 4 Units, CN: 30987
Meetings 1005 DOW, M/W: 3:00 PM – 4:30 PM (Lectures/Discussion)
1005 DOW, F: 3:00 PM – 4:00 PM (Projects/Makeups)
Instructor Mosharaf Chowdhury (ANY email regarding this course should have a title starting with “EECS582” for a timely response)
Office Hours 4820 BBB. By appointments only.
Paper Summaries Submit at http://eecs582.eecs.umich.edu/papers/

Course Description

EECS 582 will discuss advanced topics and research issues in computer and operating systems. We will take a broad view of systems research and consider common issues that emerge across individual machines to large-scale datacenters. The course schedule can broadly be divided into two tracks. In the first, we will consider the traditional computer and the challenges its operating system must address. In parallel, we will consider the datacenter as a computer, and we will observe how the designs of big data and cloud computing systems (e.g., MapReduce, Spark, HDFS) face and address many common challenges. This class will focus on identifying and understanding the enduring principles and practices in computer systems design and implementation across systems at different scale, and it will prepare students to carry out substantial independent systems research projects.

Tentative Schedule and Reading List

Date Topic Readings Lead Slides/Notes
 Sep 7 Introduction and Projects How to Read a Paper (No review) Mosharaf Slides

Project Ideas

Sep 9 No Class – Find Project Partners Writing Reviews for Systems Conferences (No review)
Sep 12 Classics The UNIX Time-Sharing System Mosharaf Slides
Sep 14 A History and Evolution of System R Mosharaf Slides
Sep 16 No Class How to Give a Bad Talk (No review)
Sep 19 Storage and File System A Case for Redundant Arrays of Inexpensive Disks (RAID) Connor Bolton Slides
Sep 21 The Design and Implementation of a Log-Structured File System

A Fast File System for UNIX (No review)

Log-structured file systems: There’s one in every SSD (No review)

Rui Chen Slides
Sep 23 No Class – Submit Proposal Draft Hints for Computer System Design (No review)
Sep 26  Kernel Exokernel: An Operating System Architecture for Application-Level Resource Management

The Multikernel: A New OS Architecture for Scalable Multicore Systems

HyunJong Lee

Jibin Yuan

Slides

Slides

Sep 28 IX: A Protected Dataplane Operating System for High Throughput and Low Latency

The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors

Jeremy Erickson

Remzi Can Aksoy

Slides

Slides

Sep 30 No Class Worse is Better (No review)
Oct 3 Virtual Memory and RPC Memory Coherence in Shared Virtual Memory Systems

Implementing Remote Procedure Calls

Andrew Kwong

Joel Sharin

Slides

Slides

Oct 5 Concurrency SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

Why Events are a Bad Idea (for High-Concurrency Servers) (No review)

Linh Nguyen Slides
Oct 7 No Class –  Finalize Project Proposal
Oct 10 Scheduling Scheduler Activations: Effective Kernel Support for the User-Level Management of Parallelism

Lottery Scheduling: Flexible Proportional-Share Resource Management

Alex Roberts

Ish Baid

Slides

Slides

Oct 12 Reliability and Fault Tolerance Improving the Reliability of Commodity Operating Systems

RaceTrack: Efficient Detection of Data Race Conditions via Adaptive Tracking

Manav Gabhawala

Michael Roberts

Slides

Slides

Oct 14 No Class End-To-End Arguments in System Design (No review)

Will The Real “End-End Principle” Please Stand Up? (No review)

Oct 17 No Class –  Fall Study Break
Oct 19 Virtualization Xen and the Art of Virtualization

Arrakis: The Operating System is the Control Plane

Timothy Trippel

Jimmy You

Slides

Slides

Oct 21 Memory Resource Management in VMware ESX Server

ReVirt: Enabling Intrusion Analysis Through Virtual-Machine Logging and Replay (No review)

Wesley Coomber Slides
Oct 24 Midterm Review  Mosharaf Slides
Oct 26 No Class – Take-Home Midterm
Oct 28 No Class How to Write a Great Research Paper (No review)
Oct 31 Datacenter as a Computer The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Second Edition (No review)

The Datacenter Needs an Operating System (No review)

 Mosharaf Slides
Nov 2 No Class – OSDI
Nov 4
Nov 7 Datacenter Programming Models MapReduce: Simplified Data Processing on Large Clusters

Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing

Haoran Wang

Peifeng Yu

Slides

Slides

Nov 9 Datacenter Operating System Large-Scale Cluster Management at Google with Borg

Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center

Arjun Khurana

Muhammed Uluyol

Slides

Slides

Nov 11 No Class – Work on Projects
Nov 14 Mid-Semester Project Presentations – 1
Nov 16 Mid-Semester Project Presentations – 2 
Nov 18 No Class – Work on Projects
Nov 21 Datacenter Resource Allocation and Scheduling Omega: Flexible, Scalable Schedulers for Large Compute Clusters

Dominant Resource Fairness: Fair Allocation of Multiple Resource Types

Altruistic Scheduling in Multi-Resource Clusters (No review)

Muhammed Uluyol

Xiao Zhu

Slides

Slides

Nov 23 Datacenter File System The Google File System

GFS: Evolution on Fast-Forward (No review)

Jiamin Huang Slides
Nov 25 No Class – Work on Projects
Nov 28 Datacenter Memory Management PACMan: Coordinated Memory Caching for Parallel Jobs

EC-Cache: Load-Balanced, Low-Latency Cluster Caching with Online Erasure Coding

Subarno Banerjee

Haoran Wang

Slides

Slides

Nov 30 No Class – Work on Projects
Dec 2 No Class – Work on Projects
Dec 5 Datacenter Networking A Scalable, Commodity Data Center Network Architecture

Efficient Coflow Scheduling with Varys

Linh Nguyen

Xiao Zhu

Slides

Slides

Dec 7  Wrap Up  Mosharaf Slides
Dec 12 Final Poster Presentation at the Tishman Hall from 3PM to 5PM

Prerequisites

The formal prerequisite for this course is EECS 482 (Introduction to Operating Systems), equivalent coursework from another university, or permission from the instructor. Courses in database systems, distributed systems, and networking are also helpful. The informal prerequisite is a strong interest in graduate systems research. You need (or need to acquire) strong technical skills to carry out a substantial design and implementation effort.

Undergrads must receive explicit permission from the instructor to enroll.

Policies

Honor Code The Engineering Honor Code applies to paper summaries and exams.

Paper Reviews Read all the papers of each day carefully and write your own paper review. Being able to critically judge others’ work is crucial for your understanding. For papers that require summaries, you must address a number of questions including:

  • What is the problem addressed by the paper, and why is this problem important?
  • What is the hypothesis of the work?
  • What is the proposed solution, and what key insight guides their solution?
  • What is one (or more) drawback or limitation of the proposal, and how will you improve it?

The paper reviews of each day must be submitted electronically at least 24 hours prior to the corresponding class. Late reviews will not be counted. You can miss up to four paper reviews during the term without penalty. Each missing review beyond the fourth one will result in 25% decrease in grade for paper reviews. Meaning, failing to turn in eight or more reviews on time will result in a zero for all paper reviews.

Reviews will be randomly picked to be graded as “weak,” “average,” and “strong.” Allocate enough time for your reading and write reviews carefully.

Paper Presentation The course will be conducted as a seminar. Each presentation should last at most 20 minutes followed by a class discussion of the designated paper(s). Each student will be assigned to present a paper at least once throughout the semester. When the number of students exceed the number of papers, some papers will be presented by a team of two students. In the presentation, you should:

  • Motivate the paper and provide background.
  • Discuss the high level idea, approach, and/or insight (using examples, whenever appropriate). Do not go into details because the other students are expected to have read them for paper summaries; focus ONLY on primary contributions.
  • Explain the difference between this paper and related work (including other readings of the same day or prior days).

The slides for your presentation must be emailed to the instructor at least 24 hours prior to the corresponding class. You should use this template for making your slides in powerpoint.

Participation You are expected to attend all lectures (you may skip up to 2 lectures due to legitimate reasons), and more importantly, participate in class discussions.

Midterm Exam There will be an open-book, take-home midterm exam based on the mandatory readings of the class up to that point in our regular meeting place. Each student must complete the exam solely by her or his own efforts. The exam must be completed within the specified time.

Final Project The largest component of the course is a class project. You will work in groups of at most three people on an instructor-approved problem and have original contribution. Surveys are not permitted as projects; instead, each project must contain a survey of background and related work. You must meet the following milestones (unless otherwise specified in the announcements section above) to ensure a high-quality project at the end of the semester:

  • Form a group of 2-3 people as soon as possible.
  • Turn in a 2-page draft proposal (including references) by September 23. Remember to include the names and Michigan email addresses of the group members. Schedule a 15-minute meeting to pitch your idea and to get early feedback.
  • Keep revising your initial idea and get more feedback until we reach an agreement. However, your team and project proposal must be finalized and approved by me on or before October 7.
  • Each group must present mid-semester progress during class hours on November 14 and November 16.
  • Each group must present their final results during a poster session on December 12.
  • Each group must turn in an 8-page final report and your code via email on or before 11:59PM EST on December 16. The report must be submitted as a PDF file, with formatting similar to that of the papers you’ve read in the class. The self-contained (i.e., include ALL dependencies) code must be submitted as a zip file. Each zip file containing the code must include a README file with a step-by-step guide on how to compile and run the provided code.

I am always available during office hours to discuss projects, and you may also make appointments at other times. It is crucial that you start working on your project early, or you are unlikely to finish in time.

Grading

Paper Reviews 20% Written summaries of 2-3 papers per week.
Paper Presentation 15% At least one paper presentation per student.
Participation 10% Preparation and participation in classroom discussions.
Midterm Exam 15% Open-book, take-home exam.
Final Project 40% Substantial research project with a final paper and poster presentation.
Honor Code

Paper Reviews

Paper Presentation

Participation

Midterm Exam

Final Project