# ows-label-audit

Implements the logic of YouTube label audits for Video Services.

Codename: Doozer/Monocle/Angler.

## Requirements

Label audit needs to have access to:
- VAPI: several calls to vapi endpoints are done to find release information.
- SQS: the application consumes 3 queues from sqs.
- Redis (via Elasticache)
- S3 (read and write)
- SNS (push and pull)

External dependencies:
- Sentry

## Install

```bash
pyvenv-3.4 env
source env/bin/activate
pip install -r requirements.txt # prod
pip install -r requirements-dev.txt  # dev
```

## Test:

```
py.test tests/
```

## Deployments

Instructions to deploy the application to qa and production:
- Add the Google keys in `keys/`, make sure those keys are properly named (`prod_google_api_key.p12` for prod, and `qa_google_api_key.p12` for qa)
- Zip the content of the application (make sure to not add in the zip the python environment and your local git.)
- Upload the zip in the ElasticBeanstalk console.

If you have new environment variables, make sure to update the EB environment first, then deploy.

## Environment Variables

```bash
export AWS_REGION=<aws_region>
export LABEL_QUEUE=<label_queue_name>
export ASSET_SEARCH_QUEUE=<asset_search_queue_name>
export AUDIT_GENERATION_RUN_INTERVAL=60
export AUDIT_GENERATION_LOCK_SECONDS=3600
export CLIENT_ID=<client_id>
export CLIENT_SECRET=<client_secret>
export BASE_URL=<base_url>
export REDIRECT_URI=<redirect_uri>
export USER_ID=<user_id>
export USER_TYPE=<user_type>
export GOOGLE_SERVICE_EMAIL=<google_service_email>
export GOOGLE_API_KEY_FILE=<google_api_key_file_path>
export EMAIL_TO_ADDRESS=<email_to_address>
export EMAIL_FROM_ADDRESS=<email_from_address>
export CACHE_ENABLED=1
export REDIS_HOST=<redis_host>
export OAUTHLIB_INSECURE_TRANSPORT=1 # needed in dev for some systems
```

## Run

### Starting Redis

```bash
redis-server
```

### Starting Flask

```bash
python application.py  # prod
python dev.py  # dev
```

### Running Workflow Tasks

The following Celery tasks perform the different stages of the prep doc generation workflow. Start each of them in different shells. Make sure you have run `source env/bin/activate` beforehand in each of the shells.

- `python -m celery -A labelaudit.tasks.label_tasks worker --loglevel=INFO`
- `python -m celery -A labelaudit.tasks.asset_search_tasks worker --loglevel=INFO`
- `python -m celery -A labelaudit.tasks.report_generation_tasks worker --loglevel=INFO --beat`

### Starting Label Audit Manually

Once you have redis, flask, and all the celery tasks running, you can kick off a label audit by doing a POST to this endpoint (assuming flask is running locally on port 5000):

`http://0.0.0.0:5000/label_audit?vendor_id=<vendor_id>&user_id=<user_id>`

You can then check the youtube_audit table for the status and path to the s3 bucket once the file is generated.

### Misc Notes

Currently, on the dev account for AWS the bucket dev-orcdbucket doesn't exist, since it exists on the prod account already. The dev account version of the bucket is dev-orchdbucket, so if you are using dev AWS credentials you'll have to change the config file to use this.