# Dae Apollo API Scrapper lambda

## Lambda should:

* be invoked by `delphi-slzApolloApiFlow` stepfunction

* get prepared by `slz_job_manager` lambda payload with UnitOfWork from event

* create and manage UoW ContentStatus

* request Apollo Starred Tracks for Apollo API

* validate API response against jsonschema

* handle validation exceptions and store them to `content_failure_log` table

* store received Starred Tracks to csv file and upload it to S3

* save upload meta information to `snaphot` table


## Lambda Environment variables
To execute the Apps Manager Lambda he following environment variables must be configured:

| Variable                 | Description                        |
|--------------------------|------------------------------------|
| `ENVIRONMENT`            | Environment name (`prod`, `dev`)   |
| `RDS_SECRETS_KEY`        | Path to SLZ DB secrets             |
| `APOLLO_API_SECRETS_KEY` | Apollo API credentials             |
| `IMPORT_BUCKET_NAME`     | Name of S3 bucket                  |
| `SENTRY_SECRET_KEY`      | Sentry secret key                  |


## Lambda Input
```json
{
    "uow_id": "apollo-20210702-sme-stream_milestones_rti-v2",
    "dsp": "apollo",
    "report_type": "stream_milestones_rti",
    "version": "v2",
    "report_date": "2021-07-02",
    "config_bucket": "dev-delphi-configs",
    "job_id": "apollo-20210702-sme-stream_milestones_rti-v2_26640_20210702T21.15.14"
}
```

## Lambda Output
In case of successful lambda execution the following output will be provided:
```json
{
    "meta": {
        "status": "OK"
    }
}
```


## Developers guide

Use `pre-commit` for git hooks managing (https://pre-commit.com/#installation). Hook configuration
can be found at .pre-commit-config.yaml. `isort` and `yapf` tools are included to run as a part
of precommit hook.

To register a new hook run:
```shell script
pre-commit install
```

To run hooks without code committing:
```shell script
pre-commit run
```

Don’t forget to run formatter check, `pylint` and `mypy` before code committing manually.
It will save you from wasting time at reading CI logs for failed build.
```shell script
make fmtcheck && make lint && make typecheck
```