How to Run Apache Kafka on Docker: A Step-by-Step Guide

Apache Kafka is a powerful open source platform that is used for building real-time data pipelines and streaming applications. Even though Kafka is highly scalable and fast, setting it up is quite time-consuming. This is where Docker comes in. Running Kafka on Docker simplifies the installation and the configuration process, which makes it easy to test locally, prototype, or even deploy lightweight environments. 

In this guide, we will learn how to run Kafka on Docker. 

Why Run Kafka on Docker?

Running Kafka on Docker has multiple benefits: 

• No need to manually install Java, Kafka, or Zookeeper. Pre-built Docker images can handle the dependencies for you. 
• Easily replicate the Kafka environments across different systems and teams. 
• Keep Kafka and Zookeeper services isolated from the host system. 
• Spin up and tear down Kafka clusters quickly for development and testing. 
• Define Kafka, Zookeeper, and other dependencies in a single YAML file for orchestration. 

Prerequisites for Running Kafka on Docker

Before you run Kafka on Docker, make sure that you have the following requirements ready. 

• Install Docker on your system (Docker Desktop for macOS/Windows, Docker Engine for Linux)
• Basic understanding of Docker commands (e.g., docker run, docker ps)

• Sufficient system resources – at least:
  • 2 GB RAM (4 GB recommended)
  • 2 CPU cores
• Optional: Docker Compose, if you prefer managing multi-container applications using YAML

Installing Docker on macOS, Linux, or Windows

Here is how you can run Kafka on Docker according to your operating system. 

macOS     

• Download the Docker Desktop installer for mac and follow the instructions. 
• After installation, run docker –version in Terminal to verify.

Windows

• Visit Docker Desktop for Windows
• Ensure WSL2 or Hyper-V is enabled.
• Install Docker and confirm using docker –version in Command Prompt or PowerShell.

Linux (Ubuntu example)

Run this command on the interface: 

sudo apt update

sudo apt install docker.io -y

sudo systemctl enable –now docker

sudo usermod -aG docker $USER

Then log out and log back in to apply user group changes.

For Docker Compose:

sudo apt install docker-compose -y

Using Docker Images for Kafka (Confluent vs Bitnami)

Feature	Confluent Kafka Image	Bitnami Kafka Image
Maintained By	Confluent (original creators of Kafka)	Bitnami (by VMware)
Image Size	Larger	Lightweight
Ease of Use	High – includes CLI tools and full platform stack	Moderate – simpler, but more manual setup required
License	Confluent Community License	Open Source (Apache 2.0)
Ideal For	Enterprise-grade deployments, feature-rich dev environments	Local development, lightweight setups, open-source projects
Components Included	Kafka, Zookeeper, Schema Registry, KSQL, Control Center	Kafka, Zookeeper (optional), minimal extras
Configuration Options	More built-in options, integrated tools	Basic Kafka/Zookeeper config via environment variables
Docker Compose Support	Official examples available	Official and community examples available
Community Support	Strong – backed by Confluent with extensive docs	Good – active GitHub community and Bitnami support

How to Install and Deploy Kafka on Docker

You can run Apache Kafka on Docker using two methods; by using the Docker CLI for a manual setup or using Docker Compose for managing multi-container applications. Here is how you can do it. 

Using Docker CLI

This method runs individual commands to start the Kafka and Zookeeper containers. 

• Pull Zookeeper and Kafka images:

docker pull bitnami/zookeeper:latest

docker pull bitnami/kafka:latest

• Start Zookeeper container:

docker run -d –name zookeeper \

  -e ALLOW_ANONYMOUS_LOGIN=yes \

  -p 2181:2181 \

  bitnami/zookeeper:latest

• Start Kafka container:

docker run -d –name kafka \

  -e KAFKA_BROKER_ID=1 \

  -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \

  -e KAFKA_LISTENERS=PLAINTEXT://:9092 \

  -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \

  -p 9092:9092 \

  –network container:zookeeper \

  bitnami/kafka:latest

However, you might need to use –network bridge and create a user-defined Docker network for better container communication in some setups.

Using Docker Compose

This is the normal approach to manage Kafka and Zookeeper together with clear configurations. 

• Create a docker-compose.yml file:
  
  version: ‘3’

services:

  zookeeper:

    image: bitnami/zookeeper:latest

    container_name: zookeeper

    ports:

      – “2181:2181”

    environment:

      – ALLOW_ANONYMOUS_LOGIN=yes

  kafka:

    image: bitnami/kafka:latest

    container_name: kafka

    ports:

      – “9092:9092”

    environment:

      – KAFKA_BROKER_ID=1

      – KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181

      – KAFKA_LISTENERS=PLAINTEXT://:9092

      – KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

• Start the containers:

docker-compose up -d

• Check if containers are running:

docker ps

Example: Docker Compose File for Kafka and Zookeeper

Here is a super simple docker-compose.yml file to run Kafka and Zookeeper using Bitnami’s official Docker images. This setup is ideal for local development and testing.

version: ‘3’

services:

  zookeeper:

    image: bitnami/zookeeper:latest

    container_name: zookeeper

    ports:

      – “2181:2181”

    environment:

      – ALLOW_ANONYMOUS_LOGIN=yes

  kafka:

    image: bitnami/kafka:latest

    container_name: kafka

    ports:

      – “9092:9092”

    environment:

      – KAFKA_BROKER_ID=1

      – KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181

      – KAFKA_LISTENERS=PLAINTEXT://:9092

      – KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092

    depends_on:

      – zookeeper

Here is how you can use this file: 

1. Save the file under the name docker-compose.yml
2. Now run the following commands to start Zookeeper and Kafka on Docker: docker-compose up -d
3. Verify if the containers are still running, using: docker ps
4. This setup will expose:

• Zookeeper on port 2181
• Kafka on port 9092

Make sure localhost:9092 works with your Kafka client. You may need to adjust the KAFKA_ADVERTISED_LISTENERS if you’re connecting from outside Docker.

How to Verify Kafka Is Running on Docker

Once Zookeeper and Kafka are up and running on Docker, you can verify the setup using a few simple steps: 

1. Check the running containers using the command: docker ps
2. Both kafka and zookeeper containers should be listed and marked as “Up”.
3. View the Kafka container logs using: docker logs kafka
4. Look for messages like started (kafka.server.KafkaServer) indicating a successful start.
5. Lastly, test the Kafka using the CLI:
   docker exec -it kafka kafka-topics.sh –bootstrap-server localhost:9092 –list
6. If Kafka is running properly, it should just list the existing topics or return nothing if no topics are created yet. 

Connecting to Kafka Inside Docker

Depending on whether the Kafka client is inside the Docker or running on your host machine, the settings might be different. 

• If you are using it from another Docker container but on the same network, use the internal service name, for example: kafka:9092.
• If you are using a local machine, use: localhost:9092 if KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 is correctly configured.
• When you are using the Kafka CLI inside the container, using 

docker exec -it kafka bash

kafka-console-producer.sh –broker-list localhost:9092 –topic test

Common Issues and Troubleshooting Tips

Issue	Possible Cause	Solution
Kafka not reachable from host machine	Misconfigured KAFKA_ADVERTISED_LISTENERS	Set PLAINTEXT://localhost:9092 properly
Zookeeper connection error	Kafka can’t find Zookeeper	Ensure Zookeeper is running and KAFKA_ZOOKEEPER_CONNECT is correct
Topic commands hang or fail	Networking or port conflict	Check port 9092 availability and Docker bridge network settings
Kafka restarts or crashes repeatedly	Invalid environment variables or missing config	Review logs with docker logs kafka to identify startup issues
Host and container network mismatch	Incorrect Docker network settings	Use depends_on and Docker Compose’s default network for container linking

Conclusion: Should You Use Kafka on Docker for Production?

Running Kafka on Docker is a really good option for development, testing, and CI workflows. It allows you a fast and repeatable setup that is super easy and needs only minimal manual instructions. 

FAQs