Experimenting with complex software systems requires practitioners to follow a list of quite onerous steps in order to get it right. Properly carrying out an experiment requires setting up infrastructure; deploying the software system under study; baselining the infrastructure; installing analysis utilities; running experiment commands and scripts iteratively; plotting and analyzing results; among others. Performing these steps manually can be cumbersome and error-prone. In this talk, I present our work on automating experimentation workflows using the Popper container-native workflow automation engine. We give an overview of workflow tools, container- and cloud-native workflow engines, and introduce Popper. To exemplify the utility of this approach, we show example workflows used in our laboratory for benchmarking and baselining Ceph, a storage management platform. Lastly, I briefly introduce and present benchmarking results for SkyhookDM, a Ceph extension that offloads computation and other data management tasks to the storage layer in order to reduce client-side resources needed for data processing in terms of CPU, memory, IO, and network traffic.
Bio: Ivo is a Research Scientist at UC Santa Cruz; an Incubator Fellow at the UC Santa Cruz Center for Research on Open Source Software (CROSS); and an Adjunct Professor at the University of Sonora (Mexico). Ivo is interested in large-scale distributed data management systems, applied aspects of data science, and reproducibility. Ivo’s 2019 PhD dissertation focused on the practical aspects in the reproducible evaluation of systems research, work for which Ivo was awarded the 2018 Better Scientific Software Fellowship. Ivo is currently working on Popper, as part of the CROSS Incubator Program.