# Refresh scripts

This project contains scripts for refreshing an RDS database from a snapshot of another database, 
potentially across AWS accounts.

## Running

### Build Image
```
$ docker-compose build base
```

### Restoring to a Database in the Same Account
Run the snapshot script to create a snapshot with the given name for the given database.
```
$ docker-compose run \
    -e DB_NAME=${SOURCE_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    refresh -m refresh.snapshot
```

Run the restore script to restore the snapshot to the database with the given name.
Data is sanitised by running the specified function.
```
docker-compose run \
    -e DB_NAME=${TARGET_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    -e SANITISE_DATA_FUNCTION_NAME=${SANITISE_DATA_FUNCTION_NAME} \
    refresh -m refresh.restore
```

### Restoring to a Database in Another Account
Run the snapshot script in the source account to create a snapshot with the given name for the given database.
```
$ docker-compose run \
    -e DB_NAME=${SOURCE_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    refresh -m refresh.snapshot
```

Run the share snapshot script in the source account to share the snapshot with the target account.
```
$ docker-compose run \
    -e DB_NAME=${SOURCE_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    -e SHARED_SNAPSHOT_NAME=${SHARED_SNAPSHOT_NAME} \
    -e SOURCE_ACCOUNT_ID=${SOURCE_ACCOUNT_ID} \
    -e SHARE_ACCOUNT_ID=${SHARE_ACCOUNT_ID} \
    -e SHARE_KMS_KEY_ID=${SHARE_KMS_KEY_ID} \
    refresh -m refresh.share_snapshot
```

Run the copy snapshot script in the target account to copy the shared snapshot to the target account.
```
$ docker-compose run \
    -e DB_NAME=${TARGET_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    -e SHARED_SNAPSHOT_NAME=${SHARED_SNAPSHOT_NAME} \
    -e SOURCE_ACCOUNT_ID=${SOURCE_ACCOUNT_ID} \
    refresh -m refresh.copy_snapshot
```

Run the restore script in the target account to restore the snapshot to the database with the given name.
Data is sanitised by running the specified function.
```
$ docker-compose run \
    -e DB_NAME=${TARGET_DB_NAME} \
    -e DB_SNAPSHOT_NAME=${DB_SNAPSHOT_NAME} \
    -e SANITISE_DATA_FUNCTION_NAME=${SANITISE_DATA_FUNCTION_NAME} \
    refresh -m refresh.restore
```


## Linting and Testing

### Run Container
```
$ docker-compose up --build lint-and-test
```

### Set pytest Options
Sometimes you may want to customize how tests run using [pytest options](https://docs.pytest.org/en/stable/usage.html) when doing development locally, only running a subset of tests using `-k` for example. Simply edit `TEST_ARGS` value in `docker-compose.yaml`, **without** quotes, to set the options you need.

:warning: Commited changes will modify how the build pipeline runs tests

### Set pytest-cov Options
By default test coverage is calculated for files in `./src` and displayed in a table after tests run.
* If you want to see a detailed output, edit `COV_REPORT` value in `docker-compose.yaml` to be `html` and visit `http://localhost:8000/`
* If you want to turn off coverage report generation, edit `COV_REPORT` value in `docker-compose.yaml` to be `off`

:warning: Commited changes will modify how the build pipeline runs tests, **do not** commit `COV_REPORT=html`

### Skip Linting
When developing locally, you might not want to worry about linting when working to make tests pass. Simple edit the `SKIP_LINT` value in `docker-compose.yaml` to be `1`.

:warning: Careful not to commit changes unless you **really** want to not run linting on the build pipeline

### Capture Exit Code
Unless the container itself crashes, or is killed, the exit code will be `0`. When the exit code of `lint-and-test.sh` script matters, a pull-request builder for example, run tests with the following flags.
```
$ docker-compose up --exit-code-from lint-and-test --abort-on-container-exit --build lint-and-test
```
### Datadog Configuration

In order to get traces and metrics (in addition to logs) from the [Datadog Lambda Library](https://docs.datadoghq.com/serverless/installation/python/?tab=containerimage), you'll need to have the following environment variables:
ENVIRONMENT
DD_LAMBDA_HANDLER (your lambda's original handler, e.g. "src.app.handler")
These can be set via terraform, for example [here](https://github.com/theorchard/terraform-infra/blob/master/qa/lambda-sound-recordings/lambda_sr_add_version.tf#L84-L91).
Notice, the linked example in terraform-infra is utilizing our terraform-lambda module, which does a lot of the setup work under the hood.