Kafka Set Up and Terminal Command For Producer And Consumer

In this blog, we learn about Kafka setup, some core concept of Kafka and some terminal command related to Kafka producer and consumer.

In this blog, we cover below command, 

a. Set up of Kafka 
b. How to start Kafka server
c. How to create a topic 
d. How to show all topic list
e. How to send some message to topic 
f. How to start consumer

Before starting the terminal command we have to understand some core concept of Kafka.

Producer:- In kafka producer is an application who send data to Kafka server.
Consumer:- In Kafka consumer is an application who read data from Kafka server.
Broker:- Producer and consumer don't communicate directly they need some one who helps them to communicate and here Kafka is that broker 
        who help them to communicate with each other.
Topic:- Topic is a unique name for Kafka stream. If producer wants to send some data first it should have some topic where to store the data
        same apply for producer it should have some topic from where to read the data.

Now, lets start our terminal command.

a. First, we have to download the Kafka, you can download it from below link. 
        -    https://kafka.apache.org/downloads

b. Extract it and open terminal

c. Kafka uses ZooKeeper so you need to first start a ZooKeeper server for this go to terminal and type below command.
        -    bin/zookeeper-server-start.sh config/zookeeper.properties

* When you start the ZooKeeper at the end it will show the port number just remember the port number, we will use this in some next command.

d. Now ZooKeeper is started after this we have to start Kafka server for this you have to type below command.
        -    bin/kafka-server-start.sh config/server.properties

e. After start Kafka server we have to create a topic with some name in which we have to store the data.
        -     bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic firstTopic

* here the first topic is the topic name
* 2181 is the port number when we start the ZooKeeper

f. You can see all topic name with the below command

        -    bin/kafka-topics.sh --list --zookeeper localhost:2181

* It will show you all the topic name which exits in the Kafka server

g. After creating topic its time to produce some data on that topic. Produce means you have to store some data on that topic
   For this, you have to type the below command

        -    bin/kafka-console-producer.sh --broker-list localhost:9092 --topic firstTopic

* By this command your producer is starting after this type message into the console to send to the server.
* 9092 is your Kafka server port number you can see this port number on the server.properties inside config folder.

h. Now it's time to consume the data which we produce by consuming we mean to read data from the server. 
   For this, you have to type command. And also open a new terminal.

        -    bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic firstTopic --from-beginning

That's it congrats you learn some important command of Kafka.