56 subscribers
با برنامه Player FM !
پادکست هایی که ارزش شنیدن دارند
حمایت شده
Towards Observability for ML Pipelines // Shreya Shankar // MLOps Coffee Sessions #75
Manage episode 318212892 series 3241972
MLOps Coffee Sessions #75 with Shreya Shankar, Towards Observability for ML Pipelines.
// Abstract
Achieving observability in ML pipelines is a mess right now. We are tracking thousands of means, percentiles, and KL divergences of features and outputs in a haphazard attempt to figure out when and how to retrain models.
In this session, we break down current unsuccessful approaches and discuss the path towards effectively maintaining ML models in production. Along the way, we introduce mltrace -- a preliminary open source project striving towards "bolt-on" observability in ML pipelines.
// Bio
Shreya Shankar is a computer scientist living in the Bay Area. She's interested in building systems to operationalize machine learning workflows. Shreya's research focus is on end-to-end observability for ML systems, particularly in the context of heterogeneous stacks of tools.
Currently, Shreya is doing her Ph.D. in the RISE lab at UC Berkeley. Previously, she was the first ML engineer at Viaduct, did research at Google Brain, and completed her BS and MS in computer science at Stanford University.
// Related Links
Shreya Shankar's blogposts: https://www.shreya-shankar.com/
Shreya Shankar's Podcasts: https://www.listennotes.com/top-episodes/shreya-shankar/
The deployment phase of machine learning by Benedict Evans: https://www.ben-evans.com/benedictevans/2019/10/4/machine-learning-deployment
Shreya Shrankar's mltrace blogpost: https://www.shreya-shankar.com/introducing-mltrace/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Shreya on LinkedIn: https://www.linkedin.com/in/shrshnk
Timestamps:
[00:00] Introduction to Shreya Shankar
[01:12] Shreya's background
[03:22] Contrast in scale influence
[05:28] Embedding ML and building machine learning infused products
[07:26] Management structure and professional incentive
[08:25] Organizational side of MLOps retros
[10:15] Tooling implementations
[12:00] Structured rational investment hardships
[13:17] Working at a start-up
[14:02] Academic work and entrepreneurial ambitions
[16:00] ML Monitoring Observability interest
[17:14] Where to get started
[20:47] Realization while at Viaduct
[23:30] Preventing alert fatigue
[27:04] Tooling bridging the gap
[30:40] Juncture at overall MLOps ecosystem
[33:58] The deployment phase of machine learning - it's the new SQL by Benedict Evans
[35:30] Model monitoring
[36:16] mltrace
[38:28] Introducing mltrace blog post series
[41:25] Tips to our content creators/writers
[43:47] Monitoring through the lens of the database
[47:37] Advice about picking up ML engineering and ML systems development in 2022
[49:36] Database low down the stack
[50:51] Most excited about 2022
[52:13] What MLOps space/ecosystem should change?
[53:21] Funding has changed the incentives around innovation
[54:52] Competition in million-dollar rounds
[55:25] Starting a company
[56:30] Wrap up
443 قسمت
Manage episode 318212892 series 3241972
MLOps Coffee Sessions #75 with Shreya Shankar, Towards Observability for ML Pipelines.
// Abstract
Achieving observability in ML pipelines is a mess right now. We are tracking thousands of means, percentiles, and KL divergences of features and outputs in a haphazard attempt to figure out when and how to retrain models.
In this session, we break down current unsuccessful approaches and discuss the path towards effectively maintaining ML models in production. Along the way, we introduce mltrace -- a preliminary open source project striving towards "bolt-on" observability in ML pipelines.
// Bio
Shreya Shankar is a computer scientist living in the Bay Area. She's interested in building systems to operationalize machine learning workflows. Shreya's research focus is on end-to-end observability for ML systems, particularly in the context of heterogeneous stacks of tools.
Currently, Shreya is doing her Ph.D. in the RISE lab at UC Berkeley. Previously, she was the first ML engineer at Viaduct, did research at Google Brain, and completed her BS and MS in computer science at Stanford University.
// Related Links
Shreya Shankar's blogposts: https://www.shreya-shankar.com/
Shreya Shankar's Podcasts: https://www.listennotes.com/top-episodes/shreya-shankar/
The deployment phase of machine learning by Benedict Evans: https://www.ben-evans.com/benedictevans/2019/10/4/machine-learning-deployment
Shreya Shrankar's mltrace blogpost: https://www.shreya-shankar.com/introducing-mltrace/
--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with Shreya on LinkedIn: https://www.linkedin.com/in/shrshnk
Timestamps:
[00:00] Introduction to Shreya Shankar
[01:12] Shreya's background
[03:22] Contrast in scale influence
[05:28] Embedding ML and building machine learning infused products
[07:26] Management structure and professional incentive
[08:25] Organizational side of MLOps retros
[10:15] Tooling implementations
[12:00] Structured rational investment hardships
[13:17] Working at a start-up
[14:02] Academic work and entrepreneurial ambitions
[16:00] ML Monitoring Observability interest
[17:14] Where to get started
[20:47] Realization while at Viaduct
[23:30] Preventing alert fatigue
[27:04] Tooling bridging the gap
[30:40] Juncture at overall MLOps ecosystem
[33:58] The deployment phase of machine learning - it's the new SQL by Benedict Evans
[35:30] Model monitoring
[36:16] mltrace
[38:28] Introducing mltrace blog post series
[41:25] Tips to our content creators/writers
[43:47] Monitoring through the lens of the database
[47:37] Advice about picking up ML engineering and ML systems development in 2022
[49:36] Database low down the stack
[50:51] Most excited about 2022
[52:13] What MLOps space/ecosystem should change?
[53:21] Funding has changed the incentives around innovation
[54:52] Competition in million-dollar rounds
[55:25] Starting a company
[56:30] Wrap up
443 قسمت
Tutti gli episodi
×
1 A Candid Conversation Around MCP and A2A // Rahul Parundekar and Sam Partee // #316 SF Live 1:04:42

1 Making AI Reliable is the Greatest Challenge of the 2020s // Alon Bochman // #312 1:01:37

1 Behavior Modeling, Secondary AI Effects, Bias Reduction & Synthetic Data // Devansh Devansh // #311 1:01:35

1 GraphBI: Expanding Analytics to All Data Through the Combination of GenAI, Graph, & Visual Analytics // Paco Nathan & Weidong Yang // #310 1:14:01

1 I Am Once Again Asking "What is MLOps?" // Oleksandr Stasyk // #308 1:07:22

1 Agents of Innovation: AI-Powered Product Ideation with Synthetic Consumer Testing // Luca Fiaschi // #306 1:02:23

1 We're All Finetuning Incorrectly // Tanmay Chopra // #304 1:00:30

1 From Rules to Reasoning Engines // George Mathew // #296 1:05:26

1 GenAI Traffic: Why API Infrastructure Must Evolve... Again // Erica Hughberg // #296 1:06:24

1 Future of Software, Agents in the Enterprise, and Inception Stage Company Building // Eliot Durbin // #293 54:26

1 The Agent Landscape - Lessons Learned Putting Agents Into Production 1:08:40

1 Evolving Workflow Orchestration // Alex Milowski // #291 1:14:34




1 Navigating Machine Learning Careers: Insights from Meta to Consulting // Ilya Reznik // #286 1:00:36

به Player FM خوش آمدید!
Player FM در سراسر وب را برای یافتن پادکست های با کیفیت اسکن می کند تا همین الان لذت ببرید. این بهترین برنامه ی پادکست است که در اندروید، آیفون و وب کار می کند. ثبت نام کنید تا اشتراک های شما در بین دستگاه های مختلف همگام سازی شود.