# Public Data Importer Lambda

This Lambda function is designed for:

- fetching a ready-to-import list of Units of Work from the Data Export API component;
- invoking a batch of Data Upsert Control components;
- fetching a list of outdated UoWs;
- updating the UoW statuses via Data Export API component.

Payload example:

```json
{
  "kwargs": {
    "exporter_api_host": "dev-exporter-api.delphiplatform.io",
    "exporter_api_schema": "http",
    "http_client_max_retries": 5,
    "http_client_request_timeout": 5,
    "http_client_retry_interval": 5,
    "import_scheduled_ttl": 43200,
    "limit": 100,
    "project": "dapd",
    "upserter_delay": 0.1
  }
}
```

## Developers guide

Use `pre-commit` for git hooks managing (https://pre-commit.com/#installation). Hook configuration
can be found at .pre-commit-config.yaml. `isort` and `yapf` tools are included to run as a part
of precommit hook.

To register a new hook run:
```shell script
pre-commit install
```

To run hooks without code committing:
```shell script
pre-commit run
```

Sometimes these tools can have a conflict regarding import sorting. In this case consider enabling
`include_trailing_comma` option at `setup.cfg` file.

Don’t forget to run `pylint` and `mypy` before code committing manually. It will save you from
wasting time at reading CI logs for failed build.

In order to check everything is fine (and have a dry run for formatting and import sorting)
consider following commands, e.g. for `dapd_exporter_lambda`:
```shell script
cd dapd_exporter_lambda
make fmtcheck
make lint
make typecheck
```

It will run all checks inside a current directory.


## Local execution

Prerequisites:

- create event you want to pass into the lambda
- put this event into `./events/<my-event>.json, they are different for DAPD, Decibel or Linkfire


Activate virtualenv

  . venv/bin/activate

Execute lambda

  python dapd_importer_lambda -e qa -p ./events/decibel.json

  usage: dapd_importer_lambda [-h] -e ENV -p EVENT_PATH
  dapd_importer_lambda: error: the following arguments are required: -e/--env, -p/--event-path