# Sales File Ingestion Daemon

### Installation

The installation is very straightforward, but make sure you are using at least
python 3.6. If you use a Mac, you can install it using `brew`. If you use
Windows, just ask for a remote VM that has it installed for you.

```bash
$ cd python-daemon
$ pyvenv env
(env) $ . env/bin/activate
(env) $ pip install -r requirements.txt
(env) $ pip install -r requirements-dev.txt
```

### Permissions

In PROD and QA environments, to accept and authorize incoming and outgoing
requests, make sure you have the right DynamoDB permissions, as highlighted
in the corresponding [tech design](https://docs.google.com/document/d/1eHoI_BddTFMi15yCaHS6KvhSSoTrMEd3WwJINIpgNpM/edit).

### Running

When all dependencies have been installed, you can run the flask application
on your local instance by running:

```bash
(env) $ python main.py
```

or

```bash
make dev
```

By using the development server, you will have access to specific features that
are not necessarily available in production, such as the exception tracer.

Note: Make sure to have your AWS credentials set up, either through environment variables or through a config file, and that the SQS queue exists.

### Testing

To run the tests, all you have to do is to run:

```bash
(env) $ py.test tests/ --cov sales --cov-report term-missing
(env) $ flake8 sales/ tests/
```

or

To run tests: `make test`

To lint: `make lint`

### Updating

To install new dependencies.

```bash
(env) $ pip install -r requirements.txt
(env) $ pip install -r requirements-dev.txt
```

or

```bash
make pip_dev
```


## API

### Message Structure
The consumed SQS message body will be valid JSON
```json
{
    "source_table_name": "dig_sales_testfile",
    "destination_s3_bucket": "prod-ows-royalties-sales-files",
    "destination_s3_path": "01-AccountingPeriodApril/03_sales.tsv",
    "sales_file_id": 3,
    "callback_url": "https://foo.bar/dag/baz"
}

```

### Response Structure
The response will be valid JSON POSTed to the callback_url
`errors` will be empty if export was successful.
/POST https://foo.bar/dag/baz
```json
{
    "source_table_name": "dig_sales_testfile",
    "destination_s3_bucket": "prod-ows-royalties-sales-files",
    "destination_s3_path": "01-AccountingPeriodApril/03_sales.tsv",
    "sales_file_id": 3,
    "callback_url": "https://foo.bar/dag/baz",
    "errors": [
        "SQSException.QueueNotFound"
    ]
}
```