Setting Up a Local Kafka Container for Your Spring Boot Application

In today’s world of microservices and event-driven architecture, Apache Kafka has become the go-to solution for streaming applications. However, setting up Kafka for local development, especially when integrating it with a Spring Boot application, can be challenging. This guide will walk you through the process of setting up a local Kafka container using Docker Desktop and integrating it with your Spring Boot application.

Why Use Kafka with Spring Boot?

Spring Boot offers seamless Kafka integration through the spring-kafka Maven package. To leverage this, you need to connect to a Kafka instance. While you could run a local Kafka instance, using Docker containers simplifies the setup process significantly. This article will guide you through the steps to set up a Kafka container and integrate it with your Spring Boot application.

Prerequisites

Before diving into the setup, ensure you have the following:

1. Docker Desktop: Install Docker Desktop by following this guide.

2. Spring Boot Application: Your Spring Boot application should have the spring-kafka package configured.

Running the Kafka Container

To run the Kafka container, you’ll need a docker-compose file. Here’s a sample configuration:

*version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ports:
- "2181:2181"

kafka:
image: confluentinc/cp-kafka:latest
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
ports:
- "9092:9092"
depends_on:
- zookeeper*

This docker-compose file pulls the Kafka container and its dependency, the Zookeeper container. Zookeeper manages Kafka broker nodes in the cluster. For more details on Zookeeper, refer to this article.

To register the containers with Docker Desktop, use the following command:

docker-compose up -d

This command pulls the required images and launches the containers. Once the containers are up, you can view them in Docker Desktop.

Creating Kafka Topics

With the Kafka container running, you can create the necessary topics using the Docker Desktop console. Use the following command to create a topic:

kafka-topics --create --topic user-notification --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092

Configuring the Spring Boot Application

Now that the Kafka container is up and running, you can launch your Spring Boot application. Configure the Kafka bootstrap address in your application.properties file as follows:

kafka.bootstrapAddress=localhost:9092

When you launch the Spring Boot application, you should see logs indicating a connection with Kafka, depending on whether your application is a producer or consumer.

Conclusion

By following these steps, you can set up a local development environment using a Kafka container and a Spring Boot application. This setup is ideal for developers working on microservices and event-driven architectures.

If you need assistance with DevOps, DevSecOps, DataOps, Cloud services, or any other aspect of microservices and infrastructure, ZippyOPS offers comprehensive consulting, implementation, and management services. Explore our services, products, and solutions. For more insights, check out our YouTube playlist. If you find this interesting, feel free to reach out to us at [email protected] for a consultation.

By following this guide, you’ll have a robust local development environment for your Spring Boot application integrated with Kafka. Happy coding!