MesosCon North America 2017

At Two Sigma, we use Cook, our open source batch scheduling mesos framework, to run millions of compute-hours of work for hundreds of users every day. This year, we are working on changing the scheduler optimization algorithm, but we want some confidence that it will actually improve utilization and the user experience (latency). Therefore, we built a mesos simulator which lets us test algorithm changes, without running the whole distributed system, and in doing so, found some subtle bugs and performance bottlenecks, and highlighted areas needing better test coverage.

In this talk, Wil will describe how we built the simulator for cook, some of the challenges in building it, and share the insights we learned by running our historical job traces through it. We will highlight the value and practicality of simulation testing for real, production systems.