Installing Apache Kafka for Production Use
Tech
2
Apache Kafka installlation involves key steps to set up a functional messaging system.
System Prerequisites
- Operating System: Kafka supports Linux, Unix-like systems, and Windows, with Linux recommended for production deployments.
- Java Runtime: Installl Java SE Development Kit (JDK) 8 or newer, as Kafka is built on Java.
Download Kafka
- Obtain the latest stable release from the Apache Kafka website (e.g., version 3.x as of 2024).
Extract the Package
- Unpack the downloaded archive into a target directory, such as
/opt/kafka.
Configure Kafka
- Modify settings in
config/server.properties, including broker identification, log storage paths, and ZooKeeper endpoints for pre-3.0 versions.
ZooKeeper Setup (Pre-Kafka 3.0)
- For older Kafka releases, install and run ZooKeeper to manage cluster coordination, following official documentation.
KRaft Mode (Kafka 3.0+)
- In Kafka 3.0 and later, KRaft mode eliminates ZooKeeper dependency. Configure internal controller election mechanisms in
config/kraft/server.properties.
Start Kafka Server
- Execute startup scripts from the Kafka directory:
# Launch a standalone server
bin/kafka-server-start.sh config/server.properties
# Start in KRaft mode
bin/kafka-server-start.sh --override advertised.listeners=PLAINTEXT://hostname:9092 config/kraft/server.properties
Create a Topic
- Use command-line utilities to establish a topic:
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic sample-topic
Verify Installation
- Test functionality by producing and consuming messages to confirm Kafka is operational.
Cluster Deployment
- For production, deploy multiple brokers across machines, configuring each with cluster details in properties files.
Security Options
- Implement optional measures like SSL/TLS encryption or SASL authentication for enhanced security in live environments.
Refer to official Kafka documantation to version-specific instructions and detailed guidance.
