.Net Core With Kafaka

What is Kafka?

Kafka is an open-source distributed event streaming platform for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. 

Kafka is publisher and consumer architecture. The publisher publishes the message on the topic, and the consumer listens to the topic.

For more details about Kafka. Please visit https://kafka.apache.org/documentation/

How to use Kafka in .Net Core

Prerequisite

• Docker for windows
• Visual Studio 2019/2022

Procedure

1.  Create a .Net web API Project named "KafkaConsumerWithDotNet"
2.  Add a yaml file.

Open a notepad, write the blow code, and save it as "docker-compose.yml" in your solution path.

version: '2'
services:
  zookeeper:
  image: wurstmeister/zookeeper
  ports:
  - "2181:2181"
  kafka:
   image: wurstmeister/kafka
   ports:
   - "9092:9092"
   environment:
 KAFKA_ADVERTISED_HOST_NAME: 127.0.0.1
 KAFKA_CREATE_TOPICS: "simpletalk_topic:1:1"
 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
   volumes:
   - /var/run/docker.sock:/var/run/docker.sock

Install confluent.kafka

You can install it using the package manager console.

PM> Install-Package Confluent.Kafka

Open cmd, navigate to the docker file inside your project and run docker-compose up.

You can see the docker is running as below. 

Fetch the code from pragnyasahoo/KafkaConsumerWithDotNet: Kafka Cosumer and Producer code using .Net Core (github.com).

Run "..\KafkaConsumerWithDotNet\KafkaConsumer" from the IDE.

Open cmd. Navigate to "..\KafkaConsumerWithDotNet\KafkaProducer" and run below command dotnet run.

Once all is set, give the request using Postman or insomnia

Then Go to the consumer IDE. You can see the message debug editor.

Happy Learning !!!