# Kafka

- Find some basic information right here -> https://docs.google.com/document/d/1N3a_sZIIF3gHMEWaZ8O8wCTyniu0-mWNfIipKWI7sYc/edit
- You can update that document with your findings & things you want to quickly lookup for.
- README of individual setup are listed below.

## Kafka Connect
- [For Elasticsearch Sink](/dparmar/kafka/elasticsearch-connect/README.md)
- [For Snowflake Sink](/dparmar/kafka/snowflake-connect/README.md)
- [To Generate Mockdata](/dparmar/kafka/datagen/README.md)

## KSQL
- [For KSQL Server](/dparmar/kafka/ksql-server/README.md)

## Topics
### Create topic
```
docker run --network=host confluentinc/cp-kafka:latest kafka-topics --create --topic label_participant_dparmar --partitions 1 --replication-factor 3 --if-not-exists --zookeeper <zookeepper_connection_url>
```

### Delete topic
```
docker run --network=host confluentinc/cp-kafka:latest kafka-topics --delete --topic label_participant_dparmar --zookeeper <zookeepper_connection_url>
```

### Get info about topic
```
docker run --network=host confluentinc/cp-kafka:latest kafka-topics --describe --topic label_participant_dparmar --zookeeper <zookeepper_connection_url>
```

### Consume topic using consumer;
```
docker run -it confluentinc/cp-kafka:latest kafka-console-consumer --bootstrap-server SSL://b-1.dev-managed-kafka-neo4.w2oo2u.c7.kafka.us-east-1.amazonaws.com:9094,SSL://b-2.dev-managed-kafka-neo4.w2oo2u.c7.kafka.us-east-1.amazonaws.com:9094,SSL://b-3.dev-managed-kafka-neo4.w2oo2u.c7.kafka.us-east-1.amazonaws.com:9094  --consumer-property security.protocol=SSL --consumer-property ssl.truststore.location=/usr/lib/jvm/zulu-8-amd64/jre/lib/security/cacerts  --topic label_participant_dparmar --from-beginning --max-messages 42
```