# OWS Analytics

## Getting Started

### Requirements

* Python3.4
* [pipenv](https://docs.pipenv.org/)

### Installation

The installation process is very straightforward. 
You will need a connection to The Orchard VPN if outside the NYC office.
The first step is to install Python 3.4 and pipenv, then get the source from 
github and install the dependencies. After that, you're ready to start playing!

* #### Install Python 3.4

  On a Mac you can use brew:
  ```
  brew install python3
  ```

  Using the Python version manager pyenv is a good alternative:
  ```
  pyenv install 3.4.6
  ```

  Then `.python-version` changes your default Python to that version.

* #### Install pipenv

  You can also use brew to install pipenv. pip can also be used directly:

  ```
  pip install --upgrade pip # this upgrades pip
  pip install pipenv
  ```

* #### Get the code
  ```bash
  mkdir analytics
  cd analytics

  git init
  git add remote orchard https://github.com/theorchard/ows-analytics.git
  git add remote fork your-fork-url
  git pull orchard master
  ```

* #### Install pinned dependencies from Pipfile.lock

  When you have the local copy, you need to install dependencies:

  ```bash
  make pip_dev
  ```

  pipenv will automatically create a virtual environment.

* #### Launch the server

  Copy `.env.shadow` to `.env` and fill in. You can use the environment 'dev'
   for testing as it will not default to 'dev'. 
   Then launch the development server with:

  ```bash
  make dev
  ```

  The labelid that will be used is set by the `LOCAL_USER` environment variable. If you want to
  run with a subaccountid, specify it with the `LOCAL_SUBACCOUNT` variable.

* #### Running tests

  We use [pytest](http://pytest.org/) to run our tests. To run unit tests:
  ```bash
  make test
  ```

  To run integration tests against the local environment, these variables need to be set in `.env` to set the `user_id`.

  ```Example
  LOCAL_USER=7123
  LOCAL_BASE_URL=http://localhost:5000
  ```

  Then run:
  ```bash
  make test_integration
  ```
  The integration tests will test requests against an actual (dev|qa) `ows-analytics` instance and queries against Snowflake.

  `test_integration` will start up the development server, run tests, and spin down the server.

  To run integration tests against the QA environment, these variables need to be set in `.env` to get the Grass token:
  ```Example
  QA_DB_USER=<YOUR_USERNAME>
  QA_DB_PASS=<YOUR_PASSWORD>
  QA_DB_HOST=<QA_DB_HOST>
  QA_DB_DATABASE=art_relations
  QA_GRASS_HOST=https://<GRASS_HOST>/auth/session
  QA_BASE_URL=https://<WORKSTATION_URL>/api/analytics
  ```
  You'll need the correct aws tokens in your ~/.aws/credentials file. The 
  best way to get them is to run the script found [here](https://github.com/theorchard/collab/tree/master/jcarrion/aws-creds-generator) 
  
  To run:
  ```bash
  ENV=QA make test_integration
  ```
    
  #### Tavern tests for access rules
  Integration tests also include [tavern](https://tavern.readthedocs.io/) tests for validation the access rules control. The tests are being executed automatically 
  when you run `make test_integration`.

* #### Running linters

  ```bash
  make lint
  ```

* #### Update the Pipfile.lock and re-install all dependencies (including dev)

  ```bash
  make install_with_dev
  ```

### Snowflake authentication using key pair

To authenticate using SSH use following documentation:
https://docs.snowflake.net/manuals/user-guide/snowsql-start.html#using-key-pair-authentication
TL;DR:
```bash
mkdir ~/.ssh/snowflake && cd ~/.ssh/snowflake # recommended
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
```
Then send newly generated public key to systems@theorchard.com and create SYS ticket.
Note: if you put key not in ~/.ssh/snowflake, then enter location in SNOWFLAKE_PRIVATE_KEY_PATH variable.

### Known issues

* #### `dev.py` still running in background after integration tests
  Possible that the `./scripts/test_integration.sh` script exited (or was killed) mid run and never killed the `dev.py` PID.