# Daemon Switchboard Consumer

### Installation

The installation is very straightforward, but make sure you are using at least
python 3.6. If you use a Mac, you can install it using `brew`. If you use
Windows, just ask for a remote VM that has it installed for you.

```bash
$ cd python-daemon
$ pyvenv env
(env) $ . env/bin/activate
(env) $ pip install -r requirements.txt
(env) $ pip install -r requirements-dev.txt
```

### Permissions

In PROD and QA environments, to accept and authorize incoming and outgoing
requests, make sure you have the right DynamoDB permissions, as highlighted
in the corresponding [tech design](https://docs.google.com/document/d/1eHoI_BddTFMi15yCaHS6KvhSSoTrMEd3WwJINIpgNpM/edit).

### Running

When all dependencies have been installed, you can run the flask application
on your local instance by running:

```bash
(env) $ python main.py
```

or

```bash
make dev
```

By using the development server, you will have access to specific features that
are not necessarily available in production, such as the exception tracer.

Note: Make sure to have your AWS credentials set up, either through environment variables or through a config file, and that the SQS queue exists.

### Testing

To run the tests, all you have to do is to run:

```bash
(env) $ py.test tests/ --cov switchboard_consumer --cov-report term-missing
(env) $ flake8 switchboard_consumer/ tests/
```

or

To run tests: `make test`

To lint: `make lint`


### Updating

To install new dependencies.

```bash
(env) $ pip install -r requirements.txt
(env) $ pip install -r requirements-dev.txt
```

or

```bash
make pip_dev
```

## Local Kafka

You may want to setup a local Kafka to test message processing for development purposes

## Steps to run Kafka full stack

- Git checkout `git@github.com:simplesteph/kafka-stack-docker-compose.git` 
- Run `docker-compose -f full-stack.yml up` to start a local Kafka cluster
- You can access the topic UI via `http://localhost:8000`
- Configure daemon-switchboard-consumer to point to the local kafka broker IP
    - `KAFKA_BOOTSTRAP_BROKERS=localhost:9092`

Note: Topics are not setup to start with. Running daemon-switchboard-consumer against the local kafka cluster will create the necessary topics.


## Post messages to Kafka

You can execute into the Kafka cluster to manually post messages into a given topic by running within `kafka-stack-docker-compose`:

```bash 
docker-compose -f full-stack.yml exec kafka1 bash
```

Execute `kafka-console-producer` against the topic you are interested in publishing messages into:
```bash
kafka-console-producer --broker-list localhost:9092 --topic enriched-updates
```

You will now be presented with a prompt to publish messages into your specified topic.

Note: minify your JSON otherwise you’ll put bad messages on the topic.

## Reading messages
To read messages posted back to kafka from the daemon, you simply need to execute `kafka-console-consumer` from within the `kafka1` container

```bash
kafka-console-consumer --bootstrap-server localhost:9092 --topic switchboard-updates
```

## Deleting topics

You may break your topics and want to clear them out

Docker exec into your Kafka container and run:
```bash
kafka-topics --zookeeper zoo1:2181 --delete --topic enriched-updates
kafka-topics --zookeeper zoo1:2181 --delete --topic switchboard-updates
```