Loading…
September 13 -14, 2017 - Los Angeles, CA
Click Here For Information & Registration
View analytic
Friday, September 15 • 2:00pm - 2:40pm
Astronomer's Journey from Cloud Lock-in to OSS Independence - Aaron Brongersma, Astronomer

Sign up or log in to save this to your schedule and see who's attending!

Astronomer is a data engineering platform that collects, processes and unifies users' data so that they can get straight to analytics and data science. Initially, Astronomer built their pipeline out of AWS-managed services. But, when they ran into the limitations of these services, they realized that they had committed to proprietary components, which prevented them from changing cloud providers.

In this talk, Astronomer outlines their transition to open source services running on Apache Mesos and DC/OS. They started by using Apache Airflow instead of AWS Simple Workflow Service and went on to replace their Kinesis streams with Apache Kafka, and deploy their stream processing applications with Marathon. They will review their remaining challenges and the solutions they are considering. It took some time and effort to for Astronomer decouple from AWS, but they are glad they did.

Speakers
avatar for Aaron Brongersma

Aaron Brongersma

VP of Engineering, NaviStone
I enjoy building and scaling infrastructure, devops and product engineering teams.



Friday September 15, 2017 2:00pm - 2:40pm
Diamond Salon 3

Attendees (19)