# JDBC Sink Connector

## Description

This type of the Kafka Connector ingests data from a kafka_topic into relational databases using corresponding JDBC driver.

### Neo4j access configuration

For local testing create and fill up `.env` file and run 
```
awsume dev
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 086679231553.dkr.ecr.us-east-1.amazonaws.com
docker-compose up --build -d
```

For the instances deployed to AWS we use Lenses' Secret Provider. This extension allows the connector to get secrets from AWS Secrets Manager.

### Example of generated connector config

[JDBC Sink Connector Configuration Properties](https://docs.confluent.io/kafka-connectors/jdbc/current/sink-connector/sink_config_options.html#jdbc-sink-connector-configuration-properties)

```json
{
    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
    "connection.url": "jdbc:mysql://phys.db.devorch.com:3306/test",

    "connection.user": "${aws:dev/kafka-connect-jdbc-sink/DATABASE_CREDENTIALS:user}",
    "connection.password": "${aws:dev/kafka-connect-jdbc-sink/DATABASE_CREDENTIALS:password}",

    "dialect.name": "MySqlDatabaseDialect",
    "auto.create": "false",
    "auto.evolve": "false",

    "insert.mode": "upsert",
    "pk.mode": "record_value",
    "pk.fields": "vendor_id",
    "fields.whitelist": "",
    "table.name.format": "${topic}",

    "transforms": "renameFirstTopic",
    "transforms.renameFirstTopic.type": "org.apache.kafka.connect.transforms.RegexRouter",
    "transforms.renameFirstTopic.regex": ".*",
    "transforms.renameFirstTopic.replacement": "mysql_table_name",

    ....

    "transforms": "renameSecondTopic",
    "transforms.renameSecondTopic.type": "org.apache.kafka.connect.transforms.RegexRouter",
    "transforms.renameSecondTopic.regex": ".*",
    "transforms.renameSecondTopic.replacement": "mysql_table_name",

    "config.providers": "aws",
    "config.providers.aws.class": "io.lenses.connect.secrets.providers.AWSSecretProvider",
    "config.providers.aws.param.aws.auth.method": "default",
    "config.providers.aws.param.aws.access.key": "dummy-client-key",
    "config.providers.aws.param.aws.secret.key": "dummy-secret-key",
    "config.providers.aws.param.aws.region": "us-east-1"
 }
```