# tiktok_openescrow

Monthly ingestion of the **TikTok OpenEscrow conflict report**.

## Source

| | |
|---|---|
| Partner | TikTok (via the orchard FTP share) |
| Frequency | Monthly |
| File | `TikTok_OpenEscrow_report_<YYYYMM>.txt.zip` (TSV, ~3.5 GB unzipped / ~340 MB zipped) |
| Drop location | `s3://prod-orcd-ftp/ftp/Royalty Reports/Musically/TikTok/` |

## Destination

`CONFLICT_REPORTING.TIKTOK.STAGING_RAW_TIKTOK_OPENESCROW`. 

## Flow

1. `get_first_day_of_month` — normalise `context_date` to the first of the month.
2. `bootstrap` — build the S3 paths for the environment.
3. `grab_drop_files` — archive the dropped `.zip` to S3 as `.gz`, or stop if the
   file has not arrived yet.
4. `set_status_to_downloaded` — set the feed status to `DOWNLOADED`.
5. `load_staging_raw_table` — stage the `.gz`, delete existing rows for the
   `download_date`, `COPY INTO` the staging_raw table, then drop the stage
   (stock `StageLoader`).
6. `set_status_to_ingested` — set the feed status to `INGESTED`.

## Schedule

Scheduled via [`Jenkinsfile`](Jenkinsfile) (`parameterizedCron`, runs daily and
relies on feed-status idempotency to pick up the monthly file whenever it
arrives). `exec.sh` derives `context_date` from `MONTHS_BACK` (default 1 → the
previous month).