# Generate Data

Scripting to hit the HIVE API, store the results in S3, and flatten the results into CSV format.

## How to use

### 1) Configure credentials

Copy `.env.example` to `.env` and fill in values.

> Ensure that you're using the API key of the correct project that's configured with the model we want to store results for.

### 2) Run the script

Ensure you have an AWS session and your user has access to the `assets` application family.

```bash
docker compose run --rm --build generate-data
```

If you need to restart the same run, set the `RUN_ID` value in `.env` before rerunning.

If the script fails, it is safe to restart with the same run ID. There are checks for check for the existing result file before hitting the HIVE API.

### 3) Inspect results

Results will be stored [in S3](https://437795906767-jjxcu2vs.us-east-1.console.aws.amazon.com/s3/buckets/prod-hive-regression-tracking?region=us-east-1&prefix=results/&showversions=false)