Cloud Computing Resource Management
Graduate Course in Cloud Computing Resource Management (6 Credits)Johan Eker
Cloud computing is rapidly transforming the ICT industry and moving compute and storage to data centers. A main driving force is cost reduction, moving CAPEX to OPEX and providing computational power in a pay-as-you-go- fashion. A main reason for the decreased cost is the improved utilization of hardware. However, main cloud vendors still report average utilization level below 30%. To improve efficiency scheduling of hardware resources is an important topic and research area. This course gives an overview to cloud computing and popular computational models while deep diving into the state of the art systems for cloud resource management.
The course is setup as a reading group where all participants are expected to have read all papers and to be able to present any given part of the current paper to the group. In addition, a cloud resource management project corresponding to one week full time work is required. The project details are defined together with the course leader on a per project basis.
------------------------------------------------------------------------
Meeting #1 - Sept 25
Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: a platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX conference on Networked systems design and implementation (NSDI'11). USENIX Association, Berkeley, CA, USA, 295-308.
http://eurosys2013.tudos.org/wp-content/uploads/2013/paper/Schwarzkopf.pdf
Meeting #2 - Oct 2
Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek, and John Wilkes. 2013. Omega: flexible, scalable schedulers for large compute clusters. In Proceedings of the 8th ACM European Conference on Computer Systems (EuroSys '13). ACM, New York, NY, USA, 351-364. DOI=10.1145/2465351.2465386 http://doi.acm.org/10.1145/2465351.2465386
Meeting #3 - Oct 9
D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr.. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the fifteenth ACM symposium on Operating systems principles (SOSP '95), Michael B. Jones (Ed.). ACM, New York, NY, USA, 251-266. DOI=10.1145/224056.224076 http://doi.acm.org/10.1145/224056.224076
Meeting #4 - Oct 21
Christina Delimitrou and Christos Kozyrakis. 2014. Quasar: resource-efficient and QoS-aware cluster management. In Proceedings of the 19th international conference on Architectural support for programming languages and operating systems (ASPLOS '14). ACM, New York, NY, USA, 127-144. DOI=10.1145/2541940.2541941 http://doi.acm.org/10.1145/2541940.2541941
Meeting #5 - Nov 6
Anshul Gandhi, Mor Harchol-Balter, Ram Raghunathan, and Michael A. Kozuch. 2012. AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers. ACM Trans. Comput. Syst. 30, 4, Article 14 (November 2012), 26 pages. DOI=10.1145/2382553.2382556 http://doi.acm.org/10.1145/2382553.2382556
Meeting #6 - Nov 13
Vinod Kumar Vavilapalli, Arun C. Murthy, Chris Douglas, Sharad Agarwal, Mahadev Konar, Robert Evans, Thomas Graves, Jason Lowe, Hitesh Shah, Siddharth Seth, Bikas Saha, Carlo Curino, Owen O'Malley, Sanjay Radia, Benjamin Reed, and Eric Baldeschwieler. 2013. Apache Hadoop YARN: yet another resource negotiator. In Proceedings of the 4th annual Symposium on Cloud Computing (SOCC '13). ACM, New York, NY, USA, , Article 5 , 16 pages. DOI=10.1145/2523616.2523633 http://doi.acm.org/10.1145/2523616.2523633
Meeting #7 - Nov 20
Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew Goldberg. 2009. Quincy: fair scheduling for distributed computing clusters. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles (SOSP '09). ACM, New York, NY, USA, 261-276. DOI=10.1145/1629575.1629601 http://doi.acm.org/10.1145/1629575.1629601
Meeting #8 - Nov 27
Hiep Nguyen, Zhiming Shen, Xiaohui Gu, Sethuraman Subbiah, and John Wilkes. 2013. AGILE: Elastic distributed resource scaling for infrastructure-as-a-service. In Proceedings of the 10th International Conference on Autonomic Computing (ICAC'13). 69--82.
https://www.usenix.org/conference/icac13/technical-sessions/presentation/nguyen
Meeting #9 - Dec 4
Rishi Kapoor, George Porter, Malveeka Tewari, Geoffrey M. Voelker, and Amin Vahdat. 2012. Chronos: predictable low latency for data center applications. In Proceedings of the Third ACM Symposium on Cloud Computing (SoCC '12). ACM, New York, NY, USA, , Article 9 , 14 pages. DOI=10.1145/2391229.2391238 http://doi.acm.org/10.1145/2391229.2391238
Meeting #10 - Dec 11
Kay Ousterhout, Patrick Wendell, Matei Zaharia, and Ion Stoica. 2013. Sparrow: distributed, low latency scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP '13). ACM, New York, NY, USA, 69-84. DOI=10.1145/2517349.2522716 http://doi.acm.org/10.1145/2517349.2522716
Meeting #11 - Dec 18 Project Presentations
Project proposals:
- Extensions to any of the more theoretical algorithms in the papers, i.e. AutoScale, Quasar, Quincy, etc.
- Evaluation and analysis of the state-of-art open source frameworks, e.g. Yarn, Hadoop, Mesos, etc.
- Analysis of commercial elasticity frameworks such as Beanstalk from Amazon and similar offerings from Google, etc.
- The results from the projects are to be presented (20 min) at the last meeting. Slides should self contained and made available to the group.