# SLZ DDEX Housekeeping lambda

## Lambda should:

- be invoked by CW event at least daily

- check for already processed releases in ContentStatus table with non-empty meta_data value

- filter COMPLETED content statuses which were processed more than COMPLETED_HOUSEKEEPING_THRESHOLD hours ago

- filter FAILED content statuses which were processed more than FAILED_HOUSEKEEPING_THRESHOLD hours ago

- delete filtered release packages from ddex-feed bucket

- update deletion mata_data of removed releases

- return removed releases list


## Lambda Environment variables
To execute the DDEX Downloader Lambda he following environment variables must be configured:

| Variable                            | Description                                       |
|-------------------------------------|---------------------------------------------------|
| `ENVIRONMENT`                       | Environment name (`prod`, `dev`)                  |
| `RDS_SECRETS_KEY`                   | Path to SLZ DB secrets                            |
| `FEED_BUCKET`                       | DDEX Feed bucket name                             |
| `COMPLETED_HOUSEKEEPING_THRESHOLD`  | Threshold for completed content statuses in hours |
| `FAILED_HOUSEKEEPING_THRESHOLD`     | Threshold for failed content statuses in hours    |
| `SENTRY_SECRET_KEY`                 | Sentry secret key                                 |


## Lambda Input
No input is required


## Lambda Output
In case of successful lambda execution the following output will be provided:
```json
{
    "meta": {
        "status": "OK",
        "deleted_releases": [
            "A10301A0000935334X_20201116181657943",
            "A10301A0001138785R_20201116181529203",
            "A10301A0001267574Q_20201116181542913",
            ...
        ]
    }
}
```


## Developers guide

Use `pre-commit` for git hooks managing (https://pre-commit.com/#installation). Hook configuration
can be found at .pre-commit-config.yaml. `isort` and `yapf` tools are included to run as a part
of precommit hook.

To register a new hook run:
```shell script
pre-commit install
```

To run hooks without code committing:
```shell script
pre-commit run
```

Sometimes these tools can have a conflict regarding import sorting. In this case consider to
enable `include_trailing_comma` option at `setup.cfg` file.

Don’t forget to run `pylint` and `mypy` before code committing manually. It will save you from
wasting time at reading CI logs for failed build.

In order to check everything is fine (and have a dry run for formatting and import sorting)
consider following commands, e.g. for `slz_ddex_housekeeping`:
```shell script
cd slz_ddex_housekeeping
workon slz_ddex_housekeeping # or './venv/bin/activate'
isort --check-only -rc . && yapf --recursive --verbose --parallel --diff . \
&& pylint --rcfile=pylintrc ./${PWD##*/} ./tests && mypy ${PWD##*/} && echo 'Exit code:' $?
```

It will run all checks inside a current directory.


## Testing

- Prepare CloudWatch event
- Put it into the `events/`
- Execute lambda:

```
    python <project_name> -e <environment> -p events/<event>.json
```