# Snowflake Sink Connector
- Sink connector to consume data from kafka & ingest it into Snowflake

## Requirements
- Make sure you have .env file with required params

## Run Setup
Start sink connector with given below command

`$ cd snowflake-connect` \
`$ docker-compose up`

## Connector Config
### How to register snowflake sink connector
- You will find given below curl request in docker-compose.yml file
- Look for  `"topics": "default_node_topic",` replace default_node_topic with topic name you want to ingest to.
- For more info on config checkout [snowflake-sink-connector-configuration-properties](https://docs.snowflake.com/en/user-guide/kafka-connector-install.html#configuring-the-kafka-connector "snowflake-sink-connector-configuration-properties")

```
curl -i -X POST -H  "Content-Type:application/json" \
        http://localhost:8083/connectors/sink_snowflake_01/config \
        -d '{
            "name": "sink_snowflake_01",
            "config": {
                "connector.class":"com.snowflake.kafka.connector.SnowflakeSinkConnector",
                "tasks.max":1,
                "topics":"default_node_topic",
                "snowflake.url.name":"$${file:/data/credentials.properties:SNOWFLAKE_HOST}",
                "snowflake.user.name":"$${file:/data/credentials.properties:SNOWFLAKE_USER}",
                "snowflake.user.role":"DEV_ENGINEERING",
                "snowflake.private.key":"$${file:/data/credentials.properties:SNOWFLAKE_PRIVATE_KEY}",
                "snowflake.private.key.passphrase":"$${file:/data/credentials.properties:SNOWFLAKE_PRIVATE_KEY_PASSPHRASE}",
                "snowflake.database.name":"DEV_ENGINEERING",
                "snowflake.schema.name":"TEST_SCHEMA",
                "key.converter":"org.apache.kafka.connect.storage.StringConverter",
                "value.converter":"com.snowflake.kafka.connector.records.SnowflakeJsonConverter"
            }
        }'
```

### How to update snowflake sink connector config
- Use curl command as given below to update config

```
curl -i -X PUT -H  "Content-Type:application/json" \
        http://localhost:8083/connectors/sink_snowflake_01/config \
        -d '{
            "connector.class":"com.snowflake.kafka.connector.SnowflakeSinkConnector",
            "tasks.max":1,
            "topics":"default_node_topic",
            "snowflake.url.name":"$${file:/data/credentials.properties:SNOWFLAKE_HOST}",
            "snowflake.user.name":"$${file:/data/credentials.properties:SNOWFLAKE_USER}",
            "snowflake.user.role":"DEV_ENGINEERING",
            "snowflake.private.key":"$${file:/data/credentials.properties:SNOWFLAKE_PRIVATE_KEY}",
            "snowflake.private.key.passphrase":"$${file:/data/credentials.properties:SNOWFLAKE_PRIVATE_KEY_PASSPHRASE}",
            "snowflake.database.name":"DEV_ENGINEERING",
            "snowflake.schema.name":"TEST_SCHEMA",
            "key.converter":"org.apache.kafka.connect.storage.StringConverter",
            "value.converter":"com.snowflake.kafka.connector.records.SnowflakeJsonConverter"
        }'
```

### How to delete snowflake sink connector
- Use curl command as given below to update sink connector

```
curl -X DELETE http://localhost:28083/connectors/sink_snowflake_01
```