32 subscribers
با برنامه Player FM !
پادکست هایی که ارزش شنیدن دارند
حمایت شده


1 How To Replace A $100,000+ Salary Within 6 MONTHS Through Buying A Small Business w/ Alex Kamenca & Carley Mitus 57:50
Decoupling with Event-Driven Architecture
Manage episode 424666725 series 2510642
In principle, data mesh architecture should liberate teams to build their systems and gather data in a distributed way, without having to explicitly coordinate. Data is the thing that can and should decouple teams, but proper implementation has its challenges.
In this episode, Kris talks to Florian Albrecht (Solution Architect, Hermes Germany) about Galapagos, an open-source DevOps software tool for Apache Kafka® that Albrecht created with his team at Hermes, a German parcel delivery company.
After Hermes chose Kafka to implement company-wide event-driven architecture, Albrecht’s team created rules and guidelines on how to use and really make the most out of Kafka. But the hands-off approach wasn’t leading to greater independence, so Albrecht’s team tried something different to documentation— they encoded the rules as software.
This method pushed the teams to stop thinking in terms of data and to start thinking in terms of events. Previously, applications copied data from one point to another, with slight changes each time. In the end, teams with conflicting data were left asking when the data changed and why, with a real impact on customers who might be left wondering when their parcel was redirected and how. Every application would then have to be checked to find out when exactly the data was changed. Event architecture terminates this cycle.
Events are immutable and changes are registered as new domain-specific events. Packaged together as event envelopes, they can be safely copied to other applications, and can provide significant insights. No need to check each application to find out when manually entered or imported data was changed—the complete history exists in the event envelope. More importantly, no more time-consuming collaborations where teams help each other to interpret the data.
Using Galapagos helped the teams at Hermes to switch their thought process from raw data to event-driven. Galapagos also empowers business teams to take charge of their own data needs by providing a protective buffer. When specific teams, providers of data or events, want to change something, Galapagos enforces a method which will not kill the production applications already reading the data. Teams can add new fields which existing applications can ignore, but a previously required field that an application could be relying on won’t be changeable.
Business partners using Galapagos found they were better prepared to give answers to their developer colleagues, allowing different parts of the business to communicate in ways they hadn’t before. Through Galapagos, Hermes saw better success decoupling teams.
EPISODE LINKS
- A Guide to Data Mesh
- Practical Data Mesh ebook
- Galapagos GitHub
- Florian Albrecht GitHub
- Watch the video
- Join the Confluent Community
- Learn more with Kafka tutorials, resources, and guides at Confluent Developer
- Live demo: Intro to Event-Driven Microservices with Confluent
- Use PODCAST100 to get $100 of free Confluent Cloud usage (details)
فصل ها
1. Intro (00:00:00)
2. What is Galapagos? (00:02:50)
3. Moving from data to events (00:04:52)
4. Enforcing rules and guidelines with Galapagos (00:09:57)
5. Business impact of using an event-driven architecture (00:19:13)
6. Galapagos and Confluent Cloud (00:23:04)
7. Challenges of moving to an event-driven architecture (00:25:32)
8. Getting started with Galapagos (00:30:20)
9. Principles of event-driven architecture (00:31:48)
10. It's a wrap! (00:37:00)
265 قسمت
Manage episode 424666725 series 2510642
In principle, data mesh architecture should liberate teams to build their systems and gather data in a distributed way, without having to explicitly coordinate. Data is the thing that can and should decouple teams, but proper implementation has its challenges.
In this episode, Kris talks to Florian Albrecht (Solution Architect, Hermes Germany) about Galapagos, an open-source DevOps software tool for Apache Kafka® that Albrecht created with his team at Hermes, a German parcel delivery company.
After Hermes chose Kafka to implement company-wide event-driven architecture, Albrecht’s team created rules and guidelines on how to use and really make the most out of Kafka. But the hands-off approach wasn’t leading to greater independence, so Albrecht’s team tried something different to documentation— they encoded the rules as software.
This method pushed the teams to stop thinking in terms of data and to start thinking in terms of events. Previously, applications copied data from one point to another, with slight changes each time. In the end, teams with conflicting data were left asking when the data changed and why, with a real impact on customers who might be left wondering when their parcel was redirected and how. Every application would then have to be checked to find out when exactly the data was changed. Event architecture terminates this cycle.
Events are immutable and changes are registered as new domain-specific events. Packaged together as event envelopes, they can be safely copied to other applications, and can provide significant insights. No need to check each application to find out when manually entered or imported data was changed—the complete history exists in the event envelope. More importantly, no more time-consuming collaborations where teams help each other to interpret the data.
Using Galapagos helped the teams at Hermes to switch their thought process from raw data to event-driven. Galapagos also empowers business teams to take charge of their own data needs by providing a protective buffer. When specific teams, providers of data or events, want to change something, Galapagos enforces a method which will not kill the production applications already reading the data. Teams can add new fields which existing applications can ignore, but a previously required field that an application could be relying on won’t be changeable.
Business partners using Galapagos found they were better prepared to give answers to their developer colleagues, allowing different parts of the business to communicate in ways they hadn’t before. Through Galapagos, Hermes saw better success decoupling teams.
EPISODE LINKS
- A Guide to Data Mesh
- Practical Data Mesh ebook
- Galapagos GitHub
- Florian Albrecht GitHub
- Watch the video
- Join the Confluent Community
- Learn more with Kafka tutorials, resources, and guides at Confluent Developer
- Live demo: Intro to Event-Driven Microservices with Confluent
- Use PODCAST100 to get $100 of free Confluent Cloud usage (details)
فصل ها
1. Intro (00:00:00)
2. What is Galapagos? (00:02:50)
3. Moving from data to events (00:04:52)
4. Enforcing rules and guidelines with Galapagos (00:09:57)
5. Business impact of using an event-driven architecture (00:19:13)
6. Galapagos and Confluent Cloud (00:23:04)
7. Challenges of moving to an event-driven architecture (00:25:32)
8. Getting started with Galapagos (00:30:20)
9. Principles of event-driven architecture (00:31:48)
10. It's a wrap! (00:37:00)
265 قسمت
Todos los episodios
×
1 Migrate Your Kafka Cluster with Minimal Downtime 1:01:30

1 Top 6 Worst Apache Kafka JIRA Bugs 1:10:58

1 Optimizing Apache JVMs for Apache Kafka 1:11:42

1 International Podcast Day - Apache Kafka Edition | Streaming Audio Special 1:02:22




1 Capacity Planning Your Apache Kafka Cluster 1:01:54




1 Streaming Analytics and Real-Time Signal Processing with Apache Kafka 1:06:33



1 Common Apache Kafka Mistakes to Avoid 1:09:43













1 Scaling an Apache Kafka Based Architecture at Therapie Clinic 1:10:56





1 The Evolution of Apache Kafka: From In-House Infrastructure to Managed Cloud Service ft. Jay Kreps 46:32



1 Expanding Apache Kafka Multi-Tenancy for Cloud-Native Systems ft. Anna Povzner and Anastasia Vela 31:01



1 From Batch to Real-Time: Tips for Streaming Data Pipelines with Apache Kafka ft. Danica Fine 29:50


















1 How to Build a Strong Developer Community with Global Engagement ft. Robin Moffatt and Ale Murray 35:18







1 Collecting Data with a Custom SIEM System Built on Apache Kafka and Kafka Connect ft. Vitalii Rudenskyi 25:14










1 Engaging Database Partials with Apache Kafka for Distributed System Consistency ft. Pat Helland 42:09

1 The Truth About ZooKeeper Removal and the KIP-500 Release in Apache Kafka ft. Jason Gustafson and Colin McCabe 31:50
















1 Building a Microservices Architecture with Apache Kafka at Nationwide Building Society ft. Rob Jackson 48:54





1 Event Streaming Trends and Predictions for 2021 ft. Gwen Shapira, Ben Stopford, and Michael Noll 44:34


1 Mastering DevOps with Apache Kafka, Kubernetes, and Confluent Cloud ft. Rick Spurgeon and Allison Walther 46:18




1 Tales from the Frontline of Apache Kafka DevOps ft. Jason Bell 1:00:25












1 Using Apache Kafka as the Event-Driven System for 1,500 Microservices at Wix ft. Natan Silnitsky 49:12




1 Disaster Recovery with Multi-Region Clusters in Confluent Platform ft. Anna McDonald and Mitch Henderson 43:04














به Player FM خوش آمدید!
Player FM در سراسر وب را برای یافتن پادکست های با کیفیت اسکن می کند تا همین الان لذت ببرید. این بهترین برنامه ی پادکست است که در اندروید، آیفون و وب کار می کند. ثبت نام کنید تا اشتراک های شما در بین دستگاه های مختلف همگام سازی شود.