## Apollo API Flow

Apollo API Flow is one of the alternative SLZ flows. All existing flows can be found in `slz-flow.json` config in `delphi-configs` S3 bucket. Сustom flow is specified in the `dsp-specific-settings.json` config as an optional `"flow"` param in implementations section.

Current flow reuses SLZ configs for SLZ UoWs creation. The `slz_job_manager` lambda is responsible for processing UoWs and step function running.

`delphi-slzApolloApiFlow` step function consists of service steps and two lambdas - `dae_apollo_api_scraper` and `slz_uow_reset` from SLZ repo. In case of any failures, notifications will be sent to Slack.

### Apollo API scrapper lambda

In this flow, we are using a new type of UoWs to get a snapshot. Such UoWs don't have complete criteria in config and don't go into completes after processing - this happens according to the rules described in `dsp_config`(for example, units go into completes at the end of the day, regardless of the results of processing).

For each unit, lambda creates a dummy `ContentStatus` to store upload status. Before the start of processing, the content status is switched to `ACTIVE` status.

To get Starred Tracks we need to get a token to access Apollo API and send a request to the Starred Tracks endpoint. API response validates against jsonchema. Validation exceptions should be stored in `content_failure_log` table.

To check snapshot updates response hash will be calculated and compared with the last hash from the snapshot table without unit binding. So if the snapshot doesn't change for several days, then UoWs during these days will not upload files to S3. If the hash has changed, then the data will be uploaded in csv format to `delphi-import` bucket. Moreover if snapshot is changed several times a day, the file will be uploaded each time. UoW and its ContentStatus will be switched to `COMPLETE` status at the end of particular period according to the rule described in `dsp_config.json`.

The meta information is written to the snapshot table and ContentStatus switchs to `MISSING` when the upload is over. UoW's status will be switched to `NOT_IN_PROGRESS` by `reset_uow_status` lambda.

### SLZ Configs

Each report type + application is a unit of work. We use SLZ configs to manage them.

*dsp_config.json example:*
```
  {
    "unit_of_work": "apollo-{yyyymmdd}-sme-stream_milestones_rti-v1",
    "uow_active_hours_threshold": 24,
    "schedule": "*/30 * * * *",
    "args": {
      "dsp": "apollo",
      "extension": "csv",
      "type": "stream_milestones_rti",
      "version": "v1",
      "licensor": "sme",
      "valid_from": "2015-01-01"
    }
  }
```

*dsp-complete-criteria.json example:*
```
  "apollo-sme-stream_milestones_rti": {
    "partner": "apollo",
    "licensor": "sme",
    "report_type": "stream_milestones_rti",
    "complete_criteria": []
  }
```

*dsp-specific-settings.json example:*
```
  "apollo": {
    "uow_active_days_limit": 14,
    "implementations": [
      {
        "rule": "sme/stream_milestones_rti",
        "label": "stream_milestones_rti",
        "flow": "apollo_api_flow"
      }
    ]
```

### Apollo API

Starred entity API returns a complete list of starred entities based on the project name(application), DSP, and entity type. It has only a full sync, there is no partial data synchronization for consistency. So we don't need to worry about historical data backfill.

First we need to get access token using API credentials from secrets and then send a request to get starred tracks.Authorization request:
```
curl --request POST \
  --url https://sme-dna.auth0.com/oauth/token \
  --header 'content-type: application/json' \
  --data '{"client_id":"CLIENT_ID","client_secret":"CLIENT_SECRET","audience":"https://api.sma.stream/","grant_type":"client_credentials"}'
```
Response:
```
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIs***","expires_in":86400,"token_type":"Bearer"}
```
Preferences request:

Params:
```
type=track
application=rti
```
```
curl --location --request GET 'https://dev-privateapi.filtr.com/user-data-api/v1/joint/favorites/?type=track&application=rti' \
--header 'Authorization: Bearer eyJhbGciOiJSUzI1NiIs***'
```
Response(list of starred ISRCs):
```
{
  "tracks":[
    "USWD11575402",
    "USRC11703646",
    "FR2X41949991",
    "GBAHT1901121"
  ],
  "playlists": null
}
```
### Snapshot DB table

Upload meta information is written to the snapshot table.
snapshot table structure:
```
snapshot_id (int)   // Primary key
content_status_id (text)  // foreign key to content_status table
file_name (text)   // upload file name
hash (text)  // md5 hash of API response
created_at (timestamptz)   // creation timestamp
```
### S3 path

Response data will be uploaded in csv format to delphi-import bucket.Output files S3 path structure:
```
s3://env-delphi-import/starred_tracks/report_type=stream_milestones/version=v1/report_date=YYYY-MM-DD/application=rti/uowid_timestamp.csv
```
file_name:  `<uow_id>_<timestamp>.csv` - UoW ID from `unit_of_work` table joined with file creation timestamp.
Output file should have next header: `isrc,application`