# tiktok_chartmetrics_loader Lambda

Flow `Ingest TikTok Scrapper URLs and update Chartmetric` consists of:
1. Get new data from Snowflake data share. Frequency should be configurable, the default interval - every 1 hour
2. Copy new records, map them to Chartmetric Ids and store them in the TIKTOK_CHARTMETRIC.APPS_ETL.TIKTOK_CHARTMETRIC_URL table
3. Send newly mapped records (and retry records failed previously) to Chartmetric API (API: Track - Add Track Links )
4. Update successfully delivered records in the  TIKTOK_CHARTMETRIC.APPS_ETL.TIKTOK_CHARTMETRIC_URL_META table

`tiktok_chartmetrics_loader` is responsible for 3,4

### Lambda Environment variables
To execute the `tiktok_chartmetrics_loader` lambda the following environment variables must be configured:

| Variable                      | Description                        
|-------------------------------|------------------------------------
| `ENVIRONMENT`                 | Environment name (`prod`, `dev`)   
| `CHARTMETRIC_SECRET_KEY`      | access to chartmetric API             
| `SNOWFLAKE_SECRETS_KEY`       | Access to snowflake DB
| `BATCH_SIZE`                  | How much records from Snowflake 
|                               | TIKTOK_CHARTMETRIC.APPS_ETL.TIKTOK_CHARTMETRIC_URL_META
|                               | table will be process during 1 lambda execution
| `SENDER_EMAIL`                | sender(to populate request to chartmetrics API)
| `REQUEST_SPACING_MS`          | (milliseconds) parameter to control the timeout between requests 
|                               | inside of the single batch to avoid rate limiting issues if they will appear
| `CHARTMETRIC_API_AUTH_URL`    | auth url to get access key from CM API
| `CHARTMETRIC_API_RECORD_URL`  | url Update Track Link from CM API
| `TEST_API`                    | CM API doesn't have dev api, only prod. For testing this param is introduced

#### TEST_API:
To use a call to PROD CM API(real calls) remove this variable from env vars.
Possible values are:
- `TEST_API: ok ' - status 200, Track was updated successfully
- `TEST_API: error_200 ' - status 200, URL is linked to another track. - Chartmetric Track ID: 18410935
- `TEST_API: error ' - status 500, Something bad happened at /api/tiktok/trackLinking. Need to check scripts.

Response will be equal for all records in batch during 1 lambda execution


### Lambda Input
See [/events](./events)


### Lambda Output
In case of successful lambda execution the following output will be provided:
```json
{
    "meta": {
      "status": "OK"
    }
}
```
In case of any generic failure we fail lambda

### Local testing

- Prepare CloudWatch event
- Put it into the `events/`
- Edit __main__.py to use added CW event(if you use IDE)
- Execute lambda:

```
    python tiktok_chartmetrics_loader -e <environment> -p <event>.json
```