# invoke-hive-ai-detection

Invoke [lambda-assets-hive-ai-detection](https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions/prod-lambda-assets-hive-ai-detection?subtab=concurrency&tab=configure) lambda function to analyze audio assets.

### Setup

1. create `.env` file that will pass Snowflake variables to the container

```bash
cp .env.shadow .env
```

2. ensure [ECR access](https://www.notion.so/AWS-Access-f841b9dd815d4443a80e96a86c92cd2f?pvs=4#7a6624c58c6642abaecd804e5f25c820) is setup
3. enable AWS access to the prod account using awsume
4. for Snowflake key-pair auth, set `SNOWFLAKE_HOST_KEY_DIR` and `SNOWFLAKE_PRIVATE_KEY_FILE_PWD` in `.env`

#### Snowflake auth Using Browser

Shorter setup, slow to run script many times.

* In `.env`
  * set `SNOWFLAKE_AUTH_TYPE` to `externalbrowser`
  * set `SNOWFLAKE_USER` to your username (should be email)
* Log into Snowflake using your browser
* Follow prompts after running script

#### Snowflake auth Using Key Pair

Longer setup, fast to run script many times.

* Setup key pairs as described [here](https://www.notion.so/Snowflake-7c88cc17b0034e7db669a88fd2962bab?pvs=4#3aa649311d19425f8a3dd2e84eda8348)
* In `.env`
  * set `SNOWFLAKE_AUTH_TYPE` to `snowflake`
  * set `SNOWFLAKE_HOST_KEY_DIR` to directory on host containing private key
  * set `SNOWFLAKE_PRIVATE_KEY_FILE_PWD` to your private key's passphrase if configured else empty
  * set `SNOWFLAKE_USER` to your username (should be email)
  
#### Manual query reference

Query for asset final data from ows-assets database:

```sql
SELECT
    af.ID,
    af.DURATION
FROM ART_RELATIONS_PROD_ART_RELATIONS.PROJECT pj
JOIN ART_RELATIONS_PROD_ART_RELATIONS.RELEASES r
    ON pj.project_id = r.PROJECT_ID
JOIN PROD_OWS_ASSETS_OWS_ASSETS.ASSET_UPLOAD au
    ON r.release_id = au.PRODUCT_ID
JOIN PROD_OWS_ASSETS_OWS_ASSETS.ASSET_FINAL af
    ON au.ID = af.ASSET_UPLOAD_ID
    AND au.IS_CORRECTION = 0
    AND au.DELETED = 0
    AND au.API_VERSION = 2
    AND af.ASSET_TYPE = 'FLAC'
    AND af.DURATION < 600 * 1000
LEFT JOIN PROD_OWS_ASSETS_OWS_ASSETS.HIVE_SEGMENT hs
    ON hs.ASSET_FINAL_ID = af.ID
WHERE
    hs.ASSET_FINAL_ID IS NULL
    --- AND pj.vendor_id = ???
    --- AND r.upc IN (???)
;
```

Run the script supports resolving asset finals directly from Snowflake for Jenkins.

### Building

* Set AWS credentials with access to ECR

```bash
docker compose build cli
```

### Running

Resolve from Snowflake by either Label ID, SUBACCOUNT_ID or PRODUCT_ID:

```
docker compose run --rm cli --env LABEL_ID=12345

docker compose run --rm cli --env ARTIST_ID=444

docker compose run --rm cli --env SUBACCOUNT_ID=789

docker compose run --rm cli --env PRODUCT_ID=456789
```

#### Full Options

options:
```
  -h, --help            show this help message and exit
  --num_workers {1,2,3,4,5,6,7,8,9,10}
                        Number of concurrent lambda invocations (default: 3)
  --batch_size BATCH_SIZE
                        Number of assets to process per lambda invocation (default: 1)
  --env {qa,prod}       Environment to use for the lambda function (default: prod)
```

### Jenkins usage

This repository can support the Jenkins job entrypoint directly through `invoke-hive-ai-detection`. The shared Jenkins job only needs to pass exactly one of `LABEL_ID`, `SUBACCOUNT_ID`, or `PRODUCT_ID`, plus the standard AWS and Snowflake environment variables.