# AWS Databricks starter
It implements integration with Databricks.

## Required permissions
* secret manager
* RDS (SLZ database)

## Lambda Environment variables
To execute the Apps Manager Lambda he following environment variables must be configured:

| Variable                    | Description                          |
|-----------------------------|--------------------------------------|
| `ENVIRONMENT`               | Environment name (`prod`, `dev`)     |
| `RDS_SECRETS_KEY`           | RDS secret key                       |
| `SENTRY_SECRET_KEY`         | Sentry secret key                    |
| `SQS_ARNS_SPLIT`            | Arn of sqs split queue               |


## Lambda Input
Lambda requires the following params: `databricks_job_id`, `sf_token`, `dsp`, `report_date` and `content_status_ids`. Example:
```json
{
    "databricks_job_id": 14,
    "sf_token": "test-step-function-token",
    "dsp": "spotify",
    "report_date": "2019-10-10",
    "content_status_ids": [12]
}
```

## Lambda Output
In case of successful lambda execution the following output will be provided:
```json
{
    "meta": {
        "status": "OK"
    }
}
```

If any kind of exception occurred, the response will look like the following:
```json
{
    "meta": {
        "status": "FAILURE",
        "message": <exception message>
    }
}
```

## Developers guide

Use `pre-commit` for git hooks managing (https://pre-commit.com/#installation). Hook configuration
can be found at .pre-commit-config.yaml. `isort` and `yapf` tools are included to run as a part
of precommit hook.

To register a new hook run:
```shell script
pre-commit install
```

To run hooks without code committing:
```shell script
pre-commit run
```

Don’t forget to run formatter check, `pylint` and `mypy` before code committing manually.
It will save you from wasting time at reading CI logs for failed build.
```shell script
make fmtcheck && make lint && make typecheck
```