■ Dione: A Framework for Automatic Profiling and Tuning Big Data Applications

Abstract.

In this demonstration we present Dione a novel framework for automatic profiling and tuning big data applications. Our system allows a non-expert user to submit Spark or Flink applications to his/her cluster and Dione automatically determines the impact of different configuration parameters on the application's execution time and monetary cost. Dione is the first framework that exploits similarities in the execution plans of different applications to narrow down the amount of profiling runs that are required for building prediction models that capture the impact of the configuration parameters on the metrics of interest. Dione exploits these prediction models to tune the configuration parameters in a way that minimizes the application's execution time or the user's budget. Finally, Dione's Web-UI visualizes the impact of the configuration parameters on the execution time and the monetary cost, and enables the user to submit the application with the recommended parameters' values.

Bibtex Entry.

@inproceedings{zacheilas2018dione,
  title={Dione: A Framework for Automatic Profiling and Tuning Big Data Applications},
  author={Zacheilas, Nikos and Maroulis, Stathis and Priovolos, Thanasis and Kalogeraki, Vana and Gunopulos, Dimitrios},
  booktitle={2018 IEEE 34th International Conference on Data Engineering (ICDE)},
  pages={1637--1640},
  year={2018},
  organization={IEEE}
}