Winning it with a multi-node Kafka in your CI | Data & AI Masters

Post Content

Streaming applications are paramount in today’s enterprise use-cases. Apache Kafka represents the industry standard event-processing platform that makes them possible. When developing a streaming application, being able to quickly create a lightweight Kafka cluster in your local machine can be very convenient for efficient development. Moreover, being able to reproduce such an environment is useful to perform integration testing of Kafka applications in CI/CD pipelines.

In this talk Canonical’s Rob Gibbon and Enrico Deusebio will demonstrate how to quickly set up a minimal, lightweight, yet fully functional, multi node Kafka deployment in your CI and how you can customise it, with just a few commands using open-source GitHub Actions. We’ll then show you how to programmatically create and manage your deployment in CI to simulate different scenarios, even with broker crashes, therefore enabling you to create more sophisticated and real-world test cases for your streaming application. Learn more at https://canonical.com/solutions/ai and https://canonical.com/data Read More Canonical Ubuntu

#linux