September 13 -14, 2017 - Los Angeles, CA
Click Here For Information & Registration
Friday, September 15 • 2:00pm - 2:40pm
Automated Performance Tuning with Bayesian Optimization - Joshua Cohen & Ramki Ramakrishna, Twitter

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Managing resource utilization is one of the hardest aspects of operating Twitter’s Mesos clusters. As the number of services grows and their resource shapes diversify, the bin packing problem becomes increasingly difficult. Tuning for optimal performance would reduce resource usage, and ease the bin packing burden. However, the multitude of available knobs, heterogeneous hardware, the large number of services, and software and hardware upgrades together make the tuning problem combinatorially intractable.

At Twitter we are developing a system that continuously performs automated tuning of services running in our Mesos clusters, using a machine learning technique called Bayesian optimization. This technique allows us to efficiently search very large parameter spaces to optimize specific performance metrics. We describe our system and share initial results.


Joshua Cohen

Twitter, Twitter
Joshua Cohen is a Senior Software Engineer at Twitter on the VM Team, working on performance optimization of JVM services. He is also a committer and PMC member for the Apache Aurora project where he has focused on deploy tooling and filesystem isolation. Previously, amongst other... Read More →

Ramki Ramakrishna

Staff Software Engineer, Twitter
Ramki Ramakrishna is a staff software engineer in the Infrastructure Engineering Division of Twitter. He is a member of the JVM Platform team and of the Twitter Architecture Group. Ramki has worked with several generations of the JVM, at Sun and Oracle, before Twitter. He has been... Read More →

Friday September 15, 2017 2:00pm - 2:40pm PDT
Diamond Salon 6