建筑 a Scale-Out Analytics Platform: YARN++ Experience

摘要

Collecting massive amounts of data and deriving business-value by running massively parallel jobs on commodity 集群 has become commonplace in the industry. 建筑 an analytics platform that provides a predictable execution substrate as well as copes with an ever-growing workload poses several interesting challenges. 使用Apache Hadoop YARN as a cluster resource management substrate, Sriram and his group, have built working systems and contributed code to YARN.

In this talk, Sriram will provide an overview of the some of the applied research 他领导的工作. Additionally, Sriram will also describe how his group was able to translate research projects into impactful code contributions to an established 开源项目.

生物

Sriram Rao works in the Data Warehouse team at Facebook Corp. Sriram is a hands-on engineer/researcher. He built KFS (Kosmos distributed filesystem), and Sailfish (scale-out distributed merge sort), and released them as  open-source projects.  Both KFS and Sailfish are deployed at Quantcast Corp backend 集群.

Prior to Facebook, he lead the Cloud and Information  Services Lab (CISL) at Microsoft.  在CISL, Sriram initiated several research projects and played a key role in shaping the  Microsoft's open source strategy around Apache YARN.  Due to his efforts, Apache YARN is widely deployed within Microsoft's  Cosmos compute 集群. Sriram has more than 20 publications in top-tier conferences such as, NSDI, OSDI, SOSP, VLDB, SIGMOD, SIGCOMM, 等. Sriram obtained his Bachelors, Masters, and Phd in Computer 科学s from University of Texas, 奥斯丁.