# core-notifications ## Notifications service ## Purpose The primary purposes of core-images service is provide a generic way to send notifications via the various providers, e.g. email, push etc. ## Project context and relations ![System context diagram](docs/context_diagram.svg) ![Container diagram](docs/container_diagram.svg) ## Configuration ### Environment variables - JSON_LOGS - LOGLEVEL - ALLOWED_HOSTS - SENTRY_DSN - CELERY_BROKER_URL - CELERY_RESULT_BACKEND - MAIL_SERVER - MAIL_PORT - MAIL_USE_TLS - MAIL_USE_SSL - MAIL_USERNAME - MAIL_PASSWORD - MAIL_DEFAULT_SENDER - ATLAS_LOGIN_URL - ATLAS_LOGOUT_URL - ATLAS_REFRESH_TOKEN_URL - ATLAS_PUBLIC_KEY_URL - FLASK_STATIC_DIGEST_HOST_URL - MEDIA_URL - MEDIA_BUCKET_NAME - MEDIA_BUCKET_PATH - AMPLITUDE_API_KEY ## Development Python package dependencies are specified under the `requirements` dir. Be sure to put only top level packages there, avoid `pip freeze`. All sub dependencies will be pinned on the docker image artifacts level. To develop core-notifications, install requirements via the `pip install -f requirements/dev.txt`. Or use `app` service from the docker-compose config for your local development needs. To test features, that are heavily rely on the Atlas auth, there are 2 possible options: - authorize on the atlas dev env, and grab the token from the `dna_bearer_token` cookie there - use the additional service `app_https` Before using the `app_https`, follow the required steps, to configure the local system: - install [Mkcert](https://github.com/FiloSottile/mkcert) in any way, suitable for your local OS - run `mkcert -install` - run `mkcert local.atlas.stream` inside the project folder to create certs - add the following record to your hosts file `127.0.0.1 local.atlas.stream` (usually `/etc/hosts`) - use the [local.atlas.stream](https://local.atlas.stream) for the app ## Code style [Flake8](https://github.com/PyCQA/flake8) linter is being used to keep the code style in a good and PEP8 compliant shape. [Black](https://github.com/psf/black) is being used for code autoformatting. Use `make docker/lint` to check code style ## Security check [Safety](https://github.com/pyupio/safety) if being used for the security check of the project dependencies. [Bandit](https://github.com/PyCQA/bandit) is being used for the security check of the project code. Use `make docker/security` to run the security checks ## Frontend Frontend styling is implemented with [Formantic UI](https://fomantic-ui.com/) framework. All raw CSS assets are located under the `assets` directory. For local development [Node.js](https://nodejs.org/uk/) is required. Install all needed dependencies with `npm install` inside the `assets` folder. Use the ```shell $ cd node_modules/fomantic-ui $ npx gulp install $ npx gulp build && npx gulp watch ``` during development produce the ready css. ## Code updates Before you push code and create a PR, run the following command: `make pre-commit` ## Attachments ### Supported types - text/csv - text/xml - text/html - text/plain - application/pdf - application/xlsx - application/json - image/gif - image/jpeg - image/png - image/svg ### How to send attachments Just get bytes of the file and decode this data like a base64 string ``` data = [['column_name_1', 'column_name_2'], ['column_value_1', 'column_value_2']] f = StringIO() csv.writer(f).writerows(data) result_bytes = f.getvalue().encode() base64_data = base64.b64encode(result_bytes).decode() ``` ## Postgres database integration Uses [sqitch](http://sqitch.org/) to manage database migrations using plain SQL commands, wrapped in several layers of integration: 1. The sqitch tool itself: https://sqitch.org/docs/manual/sqitch/ 2. An entrypoint script: [migration_entrypoint.sh](migrations/migration_entrypoint.sh) 3. A minimal Dockerfile that installs sqitch, its Postgres dependency, and the entrypoint script: [Dockerfile](./migrations/Dockerfile) 4. A Docker Compose service that defaults to deploying all migrations to the Postgres test service container: [pg_m](./docker-compose.yml) 5. A Makefile entry that builds and runs the `pg_m` service: [docker/pg/migrate](./Makefile) 6. A Makefile helper entry that adds a migration is [docker/pg/add](./Makefile) When performing local development, both forward and reverse migrations are integrated into the test suite by running `make docker/test`. The [entrypoint script](migrations/migration_entrypoint.sh) is intended to support the use case of deploying database migrations to a running database. As described in the entrypoint comments and usage message, the database to deploy to may be specified by providing a [database connection URI](https://metacpan.org/pod/URI%3A%3Adb) as an argument (e.g. `migration_entrypoint.sh -u db:pg://user:pass@host:port/name`) or by specifying the same env vars as are used by the app to configure Postgres database access: `PGDB_HOST`, `PGDB_NAME`, `PGDB_PASS`, `PGDB_PORT`, and `PGDB_USER`. The entrypoint script defaults to running `sqitch deploy`, but the command can be overridden with the `-c` option. Additional arguments to sqitch may be passed after after the argument seperator `--`. The [migration Dockerfile](./migrations/Dockerfile) just installs the entrypoint script and its dependencies, and may be controlled in the same manner, either by passing command arguments via `docker run` to the entrypoint, or by specifying the same env vars. The `pg_m` Docker Compose service is intended to be used for service coordination during automated testing and local development. It should be possible to use it to deploy migrations to external Postgres servers, but it has not been designed for that use case. Users wanting to use containers to manage migrations of live databases are directed to the [migration Dockerfile](./migrations/Dockerfile). ## Loadtest To run the loadtest for the app, run the following command: ```shell python -m loadtest \ --client_id= \ --client_secret= \ --qty_batches= \ --qty_per_batch= \ --max_requests= \ --env= \ --atlas_env= ```