---
sidebar_position: 3
---

# Producers and Consumers

Kafka clients allow you to write distributed applications and micro-services that read, write, and process streams of events in parallel, at scale, and in a fault-tolerant manner even in the case of network problems or machine failures.

This clients are categorised in **Producers** and **Consumers** and can be written in a variety of programming languages.
They are completely separate and independent of each other, meaning that producers never have to wait for consumers.

![Producers and Consumers](./img/producers_and_consumers.png)

## Producers
A Kafka Producer is a client that writes (or publishes) data or "events" into a Kafka Topic.

- An event can be structured data like JSON, unstructured text, or binary data.
- Each record sent by a producer usually consists of a key, a value, and a timestamp.
- The key is used to determine which partition within a topic the record will be sent to, this is useful for logical data ordering and grouping.
- Records can be sent to Kafka both asynchronously and synchronously.
- Producers often serialize and compress data before sending it to Kafka. Popular serialization formats include AVRO, JSON, and ProtoBuff.

When a Producer client is designed to collect data from a database or other source, it's usually referred as a **Kafka Source** client.

## Consumers
A Kafka Consumer is a client that reads (or subscribes to) Kafka Topics and consumes events from those topics.

- A consumer can subscribe to one or more Kafka topics to receive messages.
- Consumers can belong to Consumer Groups, which is a logical grouping of consumers that share a workload.
- Kafka assigns partitions of subscribed topics to consumers in a consumer group. Each partition can only be consumed by a single consumer within a group, ensuring data ordering and load distribution.
- Consumers keep track of their progress by maintaining an *offset*, which indicates the position of the last consumed record in a partition.
- Like producers, consumers often need to deserialize data from Kafka records, and the format should match the one used in the Producer.

When a Consumer client is designed to write data into a database or other target, it's usually referred as a **Kafka Sink** client.