# Apps Manager Lambda

## Lambda Environment variables
To execute the Apps Manager Lambda he following environment variables must be configured:

| Variable                        | Description                                      |
|---------------------------------|--------------------------------------------------|
| `ENVIRONMENT`                   | Environment name (`prod`, `dev`)                 |
| `CONFIG_PATH`                   | Path to config (`s3://delphi-configs/cfg.json`)  |
| `MAX_FAILURE_COUNT`             | Failed UoWs reprocessing treshold                |
| `RDS_SECRETS_KEY`               | RDS secret key                                   |
| `SF_SECRETS_KEY`                | Snowflake secret key                             |
| `BATCH_SIZE`                    | The number of processed UoWs per run             |
| `PRIORITY_DECREASE_PERIOD_DAYS` | UoWs' priority decrease period in days           |
| `SENTRY_SECRET_KEY`             | Sentry secret key                                |


## Lambda Input
No required params for lambda, but if `initialize` property is provided, manager will create UoW to current date. Also you can pass a list of data sources to process(all by default):
```json
{
    "initialize": true,
    "time": "2020-08-12T14:50:10Z",
    "data_sources": ["slz", "chartmetric"]
}
```

In case of reprocessing flow consider using `reprocessing` parameter, e.g.
```json
{
    "initialize": false,
    "time": "2020-08-12T14:50:10Z",
    "data_sources": ["slz", "chartmetric"],
    "reprocessing": [
        {
            "unit_of_work_id": 532,
            "priority": 4,
            "dbx_job_max_lifetime": 2400
        },
        {
            "unit_of_work_id": 534,
            "priority": 8
        }
    ]
}
```

If the `--driver-java-options` for `dbx_spark_params` update is required, add `add_driver_java_options` key to payload:
```json
{
    "initialize": false,
    "time": "2020-08-12T14:50:10Z",
    "data_sources": ["slz", "chartmetric"],
    "reprocessing": [
        {
            "unit_of_work_id": 532,
            "priority": 4
        }
    ],
    "add_driver_java_options": ["-Doption1", "-Doption2"]
}
```

If the `--driver-java-options` for `dbx_spark_params` full overridding is required,
add `override_driver_java_options` key to payload:
```json
{
    "initialize": false,
    "time": "2020-08-12T14:50:10Z",
    "data_sources": ["slz", "chartmetric"],
    "reprocessing": [
        {
            "unit_of_work_id": 532,
            "priority": 4
        }
    ],
    "override_driver_java_options": ["option1", "option2"]
}
```

Both `add_driver_java_options` and `override_driver_java_options` are available only with `reprocessing` option.

## Lambda Output
In case of successful lambda execution the following output will be provided:
```json
{
    "meta": {
        "status": "OK"
    }
}
```

If any kind of exception occurred, the response will look like the following:
```json
{
    "meta": {
        "status": "FAILURE",
        "message": <exception message>
    }
}
```

## Developers guide

Use `pre-commit` for git hooks managing (https://pre-commit.com/#installation). Hook configuration
can be found at .pre-commit-config.yaml. `isort` and `yapf` tools are included to run as a part
of precommit hook.

To register a new hook run:
```shell script
pre-commit install
```

To run hooks without code committing:
```shell script
pre-commit run
```

Don’t forget to run formatter check, `pylint` and `mypy` before code committing manually.
It will save you from wasting time at reading CI logs for failed build.
```shell script
make fmtcheck && make lint && make typecheck
```