# Spotify Stream Benchmarking

[wiki](https://data-analytics.atlassian.net/wiki/spaces/DS/pages/67535017/Benchmark+by+Streams+Findings)

## Notebooks

[Streaming Data analytics (Isabel Barry)](https://github.com/filtr/ds-stream-benchmarking/blob/develop/notebooks/Stream%20Benchmarking.ipynb)

## CLI

To use the CLI, make sure you either use the venv enclosed, or have all 
files in requirements.txt installed (`pip3 install -r requirements.txt`).

### Basic usage

You **must** be able to access Reporting DB from the box you are running
this on.

In interactive mode, the minimum you must provide is a destination CSV
file where to save the results. To do so, call:

`python3 cli.py --destination quantiles.csv`

This will log into the Redshift server, obtain the data, perform 
the quantile calculation and save the results to quantiles.csv. It will
ask you for your username and password interactively, and the password 
entry is obscured, so it will not be recorded in your shell history.

### Non-interactive mode

For non-interactive mode, e.g. via Docker, you can pass in `--username`
and `--password` for the database.

### Other parameters

You may also specify other parameters. `python3 cli.py --help` lists 
them:

```
Usage: cli.py [OPTIONS]

Options:
  --username TEXT     Your Reporting DB server username
  --password TEXT     Your Reporting DB server password
  --hostname TEXT     Reporting DB server host name, defaults to reporting-
                      db.smeanalyticsapps.com
  --dbname TEXT       The database to query from, defaults to smaredshiftdb
  --port TEXT         Port to access the DB from, defaults to Redshift default
                      5432
  --destination TEXT  Path to CSV file to write results to
  --help              Show this message and exit.
```