Latest Event stream processing Podcast Episodes
Apache Kafka 3.5 - Kafka Core, Connect, Streams, & Client Updates
Streaming Audio: Apache Kafka® & Real-Time Data - June 15, 2023 16:00 - 11 minutes ★★★★★ - 33 ratingsApache Kafka® 3.5 is here with the capability of previewing migrations between ZooKeeper clusters to KRaft mode. Follow along as Danica Fine highlights key release updates. Kafka Core: KIP-833 provides an updated timeline for KRaft. KIP-866 now is preview and allows migration from an existing...
A Special Announcement from Streaming Audio
Streaming Audio: Apache Kafka® & Real-Time Data - April 13, 2023 07:00 - 1 minute ★★★★★ - 33 ratingsAfter recording 64 episodes and featuring 58 amazing guests, the Streaming Audio podcast series has amassed over 130,000 plays on YouTube in the last year. We're extremely proud of these achievements and feel that it's time to take a well-deserved break. Streaming Audio will be taking a vacation...
How to use Data Contracts for Long-Term Schema Management
Streaming Audio: Apache Kafka® & Real-Time Data - March 21, 2023 07:00 - 57 minutes ★★★★★ - 33 ratingsHave you ever struggled with managing data long term, especially as the schema changes over time? In order to manage and leverage data across an organization, it’s essential to have well-defined guidelines and standards in place around data quality, enforcement, and data transfer. To get started...
How to use Python with Apache Kafka
Streaming Audio: Apache Kafka® & Real-Time Data - March 14, 2023 07:00 - 31 minutes ★★★★★ - 33 ratingsCan you use Apache Kafka® and Python together? What’s the current state of Python support? And what are the best options to get started? In this episode, Dave Klein joins Kris to talk about all things Kafka and Python: the libraries, the tools, and the pros & cons. He also talks about the new co...
Next-Gen Data Modeling, Integrity, and Governance with YODA
Streaming Audio: Apache Kafka® & Real-Time Data - March 07, 2023 08:00 - 55 minutes ★★★★★ - 33 ratingsIn this episode, Kris interviews Doron Porat, Director of Infrastructure at Yotpo, and Liran Yogev, Director of Engineering at ZipRecruiter (formerly at Yotpo), about their experiences and strategies in dealing with data modeling at scale. Yotpo has a vast and active data lake, comprising thous...
Migrate Your Kafka Cluster with Minimal Downtime
Streaming Audio: Apache Kafka® & Real-Time Data - March 01, 2023 08:00 - 1 hour ★★★★★ - 33 ratingsMigrating Apache Kafka® clusters can be challenging, especially when moving large amounts of data while minimizing downtime. Michael Dunn (Solutions Architect, Confluent) has worked in the data space for many years, designing and managing systems to support high-volume applications. He has helpe...
Real-Time Data Transformation and Analytics with dbt Labs
Streaming Audio: Apache Kafka® & Real-Time Data - February 22, 2023 08:00 - 43 minutes ★★★★★ - 33 ratingsdbt is known as being part of the Modern Data Stack for ELT processes. Being in the MDS, dbt Labs believes in having the best of breed for every part of the stack. Oftentimes folks are using an EL tool like Fivetran to pull data from the database into the warehouse, then using dbt to manage the ...
What is the Future of Streaming Data?
Streaming Audio: Apache Kafka® & Real-Time Data - February 15, 2023 08:00 - 41 minutes ★★★★★ - 33 ratingsWhat’s the next big thing in the future of streaming data? In this episode, Greg DeMichillie (VP of Product and Solutions Marketing, Confluent) talks to Kris about the future of stream processing in environments where the value of data lies in their ability to intercept and interpret data. Greg...
What can Apache Kafka Developers learn from Online Gaming?
Streaming Audio: Apache Kafka® & Real-Time Data - February 08, 2023 08:00 - 55 minutes ★★★★★ - 33 ratingsWhat can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern...
Apache Kafka 3.4 - New Features & Improvements
Streaming Audio: Apache Kafka® & Real-Time Data - February 07, 2023 16:00 - 5 minutes ★★★★★ - 33 ratingsApache Kafka® 3.4 is released! In this special episode, Danica Fine (Senior Developer Advocate, Confluent), shares highlights of the Apache Kafka 3.4 release. This release introduces new KIPs in Kafka Core, Kafka Streams, and Kafka Connect. In Kafka Core: KIP-792 expands the metadata each grou...
How to use OpenTelemetry to Trace and Monitor Apache Kafka Systems
Streaming Audio: Apache Kafka® & Real-Time Data - February 01, 2023 08:00 - 50 minutes ★★★★★ - 33 ratingsHow can you use OpenTelemetry to gain insight into your Apache Kafka® event systems? Roman Kolesnev, Staff Customer Innovation Engineer at Confluent, is a member of the Customer Solutions & Innovation Division Labs team working to build business-critical OpenTelemetry applications so companies c...
What is Data Democratization and Why is it Important?
Streaming Audio: Apache Kafka® & Real-Time Data - January 26, 2023 08:00 - 47 minutes ★★★★★ - 33 ratingsData democratization allows everyone in an organization to have access to the data they need, and the necessary tools needed to use this data effectively. In short, data democratization enables better business decisions. In this episode, Rama Ryali, a Senior IT and Data Executive, chats with K...
Git for Data: Managing Data like Code with lakeFS
Streaming Audio: Apache Kafka® & Real-Time Data - January 19, 2023 08:00 - 30 minutes ★★★★★ - 33 ratingsIs it possible to manage and test data like code? lakeFS is an open-source data version control tool that transforms object storage into Git-like repositories, offering teams a way to use the same workflows for code and data. In this episode, Kris sits down with guest Adi Polak, VP of DevX at Tr...
Using Kafka-Leader-Election to Improve Scalability and Performance
Streaming Audio: Apache Kafka® & Real-Time Data - January 12, 2023 08:00 - 51 minutes ★★★★★ - 33 ratingsHow does leader election work in Apache Kafka®? For the past 2 ½ years, Adithya Chandra, Staff Software Engineer at Confluent, has been working on Kafka scalability and performance, specifically partition leader election. In this episode, he gives Kris Jenkins a deep dive into the power of leade...
Real-Time Machine Learning and Smarter AI with Data Streaming
Streaming Audio: Apache Kafka® & Real-Time Data - January 05, 2023 08:00 - 38 minutes ★★★★★ - 33 ratingsAre bad customer experiences really just data integration problems? Can real-time data streaming and machine learning be democratized in order to deliver a better customer experience? Airy, an open-source data-streaming platform, uses Apache Kafka® to help business teams deliver better results t...
The Present and Future of Stream Processing
Streaming Audio: Apache Kafka® & Real-Time Data - December 28, 2022 08:00 - 31 minutes ★★★★★ - 33 ratingsThe past year saw new trends emerge in the world of data streaming technologies, as well as some unexpected and novel use cases for Apache Kafka®. New reflections on the future of stream processing and when companies should adopt microservice architecture inspired several talks at this year’s in...
Top 6 Worst Apache Kafka JIRA Bugs
Streaming Audio: Apache Kafka® & Real-Time Data - December 21, 2022 08:00 - 1 hour ★★★★★ - 33 ratingsEntomophiliac, Anna McDonald (Principal Customer Success Technical Architect, Confluent) has seen her fair share of Apache Kafka® bugs. For her annual holiday roundup of the most noteworthy Kafka bugs, Anna tells Kris Jenkins about some of the scariest, most surprising, and most enlightening cor...
Learn How Stream-Processing Works The Simplest Way Possible
Streaming Audio: Apache Kafka® & Real-Time Data - December 20, 2022 08:00 - 31 minutes ★★★★★ - 33 ratingsCould you explain Apache Kafka® in ways that a small child could understand? When Mitch Seymour, author of Mastering Kafka Streams and ksqlDB, wanted a way to communicate the basics of Kafka and event-based stream processing, he decided to author a children’s book on the subject, but it turned i...
Building and Designing Events and Event Streams with Apache Kafka
Streaming Audio: Apache Kafka® & Real-Time Data - December 15, 2022 08:00 - 53 minutes ★★★★★ - 33 ratingsWhat are the key factors to consider when developing event-driven architecture? When properly designed, events can connect existing systems with a common language and allow data exchange in near real time. They also help reduce complexity by providing a single source of truth that eliminates the...
Rethinking Apache Kafka Security and Account Management
Streaming Audio: Apache Kafka® & Real-Time Data - December 08, 2022 08:00 - 41 minutes ★★★★★ - 33 ratingsIs there a better way to manage access to resources without compromising security? New employees need access to a variety of resources within a company's tech stack. But manually granting access can be error-prone. And when employees leave, their access must be revoked, thus potentially introduc...
Real-time Threat Detection Using Machine Learning and Apache Kafka
Streaming Audio: Apache Kafka® & Real-Time Data - November 29, 2022 08:00 - 29 minutes ★★★★★ - 33 ratingsCan we use machine learning to detect security threats in real-time? As organizations increasingly rely on distributed systems, it is becoming more important to analyze the traffic that passes through those systems quickly. Confluent Hackathon ’22 finalist, Géraud Dugé de Bernonville (Data Consu...
Improving Apache Kafka Scalability and Elasticity with Tiered Storage
Streaming Audio: Apache Kafka® & Real-Time Data - November 22, 2022 08:00 - 29 minutes ★★★★★ - 33 ratingsWhat happens when you need to store more than a few petabytes of data? Rittika Adhikari (Software Engineer, Confluent) discusses how her team implemented tiered storage, a method for improving the scalability and elasticity of data storage in Apache Kafka®. She also explores the motivating facto...
Improving Kafka Scalability and Elasticity with Tiered Storage
Streaming Audio: Apache Kafka® & Real-Time Data - November 22, 2022 08:00 - 29 minutes ★★★★★ - 33 ratingsWhat happens when you need to store more than a few petabytes of data? Rittika Adhikari (Software Engineer, Confluent) discusses how her team implemented tiered storage, a method for improving the scalability and elasticity of data storage in Apache Kafka®. She also explores the motivating facto...
Decoupling with Event-Driven Architecture
Streaming Audio: Apache Kafka® & Real-Time Data - November 15, 2022 08:00 - 38 minutes ★★★★★ - 33 ratingsIn principle, data mesh architecture should liberate teams to build their systems and gather data in a distributed way, without having to explicitly coordinate. Data is the thing that can and should decouple teams, but proper implementation has its challenges. In this episode, Kris talks to Flo...
If Streaming Is the Answer, Why Are We Still Doing Batch?
Streaming Audio: Apache Kafka® & Real-Time Data - November 09, 2022 08:00 - 43 minutes ★★★★★ - 33 ratingsIs real-time data streaming the future, or will batch processing always be with us? Interest in streaming data architecture is booming, but just as many teams are still happily batching away. Batch processing is still simpler to implement than stream processing, and successfully moving from batc...
Security for Real-Time Data Stream Processing with Confluent Cloud
Streaming Audio: Apache Kafka® & Real-Time Data - November 03, 2022 07:00 - 48 minutes ★★★★★ - 33 ratingsStreaming real-time data at scale and processing it efficiently is critical to cybersecurity organizations like SecurityScorecard. Jared Smith, Senior Director of Threat Intelligence, and Brandon Brown, Senior Staff Software Engineer, Data Platform at SecurityScorecard, discuss their journey fro...
Running Apache Kafka in Production
Streaming Audio: Apache Kafka® & Real-Time Data - October 27, 2022 07:00 - 58 minutes ★★★★★ - 33 ratingsWhat are some recommendations to consider when running Apache Kafka® in production? Jun Rao, one of the original Kafka creators, as well as an ongoing committer and PMC member, shares the essential wisdom he's gained from developing Kafka and dealing with a large number of Kafka use cases. Here...
Build a Real Time AI Data Platform with Apache Kafka
Streaming Audio: Apache Kafka® & Real-Time Data - October 20, 2022 07:00 - 37 minutes ★★★★★ - 33 ratingsIs it possible to build a real-time data platform without using stateful stream processing? Forecasty.ai is an artificial intelligence platform for forecasting commodity prices, imparting insights into the future valuations of raw materials for users. Nearly all AI models are batch-trained once,...
Optimizing Apache JVMs for Apache Kafka
Streaming Audio: Apache Kafka® & Real-Time Data - October 13, 2022 07:00 - 1 hour ★★★★★ - 33 ratingsJava Virtual Machines (JVMs) impact Apache Kafka® performance in production. How can you optimize your event-streaming architectures so they process more Kafka messages using the same number of JVMs? Gil Tene (CTO and Co-Founder, Azul) delves into JVM internals and how developers and architects ...
Apache Kafka 3.3 - KRaft, Kafka Core, Streams, & Connect Updates
Streaming Audio: Apache Kafka® & Real-Time Data - October 03, 2022 15:00 - 6 minutes ★★★★★ - 33 ratingsApache Kafka® 3.3 is released! With over two years of development, KIP-833 marks KRaft as production ready for new AK 3.3 clusters only. On behalf of the Kafka community, Danica Fine (Senior Developer Advocate, Confluent) shares highlights of this release, with KIPs from Kafka Core, Kafka Stream...
Related Event stream processing Topics