Building web-scale observability at Slack, Pinterest & Twitter - OpenObservability Talks S2E09

OpenObservability Talks

What does it take to build observability in a web-scale company such as Slack, Pinterest and Twitter?

On this episode of OpenObsevability Talks I'll host Suman Karumuri to hear how he built these systems from the ground up on these #BigTech co's, about his recent research papers and more.

Suman Karumuri is a Sr. Staff Software Engineer and the tech lead for Observability at Slack. Suman Karumuri is an expert in distributed tracing and was a tech lead of Zipkin and a co-author of OpenTracing standard, a Linux Foundation project via the CNCF. Previously, Suman Karumuri has spent several years building and operating petabyte scale log search, distributed tracing and metrics systems at Pinterest, Twitter and Amazon. In his spare time, he enjoys board games, hiking and playing with his kids.

The episode was live-streamed on 16 February 2022 and the video is available at https://youtu.be/IvidkV3TfYg

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Show Notes:

* Who owns observability in large organizations?

* The gaps in current way of handling metrics

* MACH research paper for metrics storage engine

* The gaps in current way of handling logs Slack KalDB

* SlackTrace - Slack in house tracing system

Resources:

• Research paper: building Observability Data Management Systems
• CIDR paper: Video
• SlackTrace blog post, talk.
• Logging at Twitter
• Pintrace: A Distributed Tracing Pipeline talk by Suman at LISA
• Observability Engineering book
• Observability Trends for 2022
• Yelp engineering with Elasticsearch and Lucene

Socials:

• Twitter: https://twitter.com/OpenObserv
• Twitch: https://www.twitch.tv/openobservability
• YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg