# pdp-backfill
PP Tools for backfilling to Identities/Roles application_family: permissions-platform

## Setup
This application uses Poetry for dependency management. If you're planning on doing development locally you'll need to ensure [Poetry is installed](https://python-poetry.org/docs/#installation).

This application uses Makefile for common functionality. Go ahead and try it out:

```sh
make clean env lint test_unit
```

The first time you may need to run
```sh
poetry run pre-commit install
```
to install the pre-commit hooks.

## Generating Models for the ows-pdp client

We use `openapitools/openapi-generator` to generate models for
interacting with relevant ows-pdp endpoints. To re-generate models
using qa-ows-pdp.theorchard.io, run:

🚨 You will likely want to delete the `backfill/connectors/ows_pdp/models` dir before 
re-generating models! `make generate_models` will not delete old, no longer used models, it will simply add new models and replace existing models. 
Ensure that you still have a backfill/connectors/ows_pdp/models/__init__.py file, however! 🚨

```sh
make generate_models
```
You can review the full list of files that were autogenerated in `backfill/connectors/.openapi-generator/FILES`.

If the `make generate_models` command generates models which should be excluded (e.g. the model exposes PDP functionality), you can:
* Discard all changes: `git checkout -- .` and remove all new files as well as any existing files you don't want.
* Add the file path or wildcard to ignore to `.openapi-generator-ignore`
* Re-run `make generate_models`

** Keep in mind this tooling does not remove files that are already committed. You have to do that manually. **

## Running on ECS

Update your `~/.aws/config` with something like:
```
[profile permissions-platform-qa]
source_profile = prod
role_session_name = <your username>
role_arn = arn:aws:iam::591204808501:role/permissions-platform-role

[profile permissions-platform-qa-pdp-backfill-run-task-role]
source_profile = permissions-platform-qa
role_arn = arn:aws:iam::591204808501:role/qa-pdp-backfill-task-run-role
```

Then run:
```
awsume permissions-platform-qa-pdp-backfill-run-task-role

aws ecs run-task \
--cluster arn:aws:ecs:us-east-1:591204808501:cluster/qa-pdp-backfill \
--count 1 \
--task-definition arn:aws:ecs:us-east-1:591204808501:task-definition/qa-pdp-backfill \
--launch-type FARGATE \
--network-configuration 'awsvpcConfiguration={subnets=["subnet-028ea50ff1880724d","subnet-0b463a07523ed2058","subnet-05ed096fc8769f253"],securityGroups=["sg-0eb7553dd0d95f090"],assignPublicIp="DISABLED"}' \
--overrides '{"containerOverrides": [
    {"name": "pdp-backfill", "command": ["backfill", "process", "--bucket-name", "qa-pdp-backfill", "--manifest-file", "jchung/pdp_qa_refresh/manifest.json"]}
]}' \
--tags='[{"key": "environment","value": "qa"},{"key": "service_name","value": "pdp-backfill"}]'
```