![Streaming Audio: Apache Kafka® & Real-Time Data artwork](https://is3-ssl.mzstatic.com/image/thumb/Podcasts123/v4/10/7e/fb/107efb84-c93f-ac4c-0224-70cc25eec78a/mza_1926486998700164096.jpg/100x100bb.jpg)
Distributed Systems Engineering with Apache Kafka ft. Guozhang Wang
Streaming Audio: Apache Kafka® & Real-Time Data
English - January 25, 2021 14:00 - 44 minutes - 61.7 MB - ★★★★★ - 33 ratingsTechnology Education How To apache kafka confluent event stream processing cloud real-time data event driven architecture open source data in motion kafka cloud native Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed
Tim Berglund picks the brain of a distributed systems engineer, Guozhang Wang, tech lead in the Streaming department of Confluent. Guozhang explains what compelled him to join the Stream Processing team at Confluent coming from the Apache Kafka® core infrastructure. He reveals what makes the best distributed systems infrastructure engineers tick and how to prepare to take on this kind of role—solving failure scenarios, a satisfying challenge.
One challenge in distributed systems is achieving agreements from multiple nodes that are connected in a Kafkacluster, but the connection in practice is asynchronous.
Guozhang also shares the newest updates in the Kafka community, including the coming ZooKeeper-free architecture where metadata will be maintained by Kafka logs.
Prior to joining Confluent, Guozhang worked for LinkedIn, where he used Kafka for a few years before he started asking himself, “How fast can I get value from the data that I’ve collected?” This question eventually led him to begin building Kafka Streams and ksqlDB. Ever since, he’s been working to advance stream processing, and in this episode, provides an exciting preview of what’s to come.
EPISODE LINKS
Join the Confluent teamDiving into Exactly-Once Semantics with Guozhang WangIn Search of an Understandable Consensus AlgorithmThe Curious Incident of the State Store in Recovery in ksqlDBFrom Eager to Smarter in Apache Kafka Consumer RebalancesKIP-595: A Raft Protocol for the Metadata QuorumJoin the Confluent Community SlackLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperLive demo: Kafka streaming in 10 minutes on Confluent CloudUse 60PDCAST to get an additional $60 of free Confluent Cloud usage (details)