# atlas-um
## User account management for SME Data & Analytics

## Purpose

The primary purposes of atlas-um is to provide a unified
authentication portal and authorization hub to the apps developed by
Sony Music Entertainment’s Data & Analytics group. At present, DnA is
transitioning from Auth0 to USM for authentication, and atlas-um’s
secondary goal is to ease and speed that transition.

## Project context and relations

![System context diagram](docs/context_diagram.svg)

![Container diagram](docs/container_diagram.svg)

## Configuration

### Environment variables
  Refer the [.env.sample](.env.sample) file for the possible configuration options


## Development

Make sure that you logged in to AWS ECR by running `make docker/login/base`

To develop atlas-um run `make docker/run/local` to bootstrap the
Docker development environment. 

To develop atlas-um locally without the Docker container install requirements via the `pip install -f requirements/dev.txt`.

Python package dependencies are specified under the `requirements` dir.
Be sure to put only top level packages there, avoid `pip freeze`.
All sub dependencies will be pinned on the docker image artifacts level.

### Unit tests

Pytest is being used for unit testing.
When unit testing the functionality of atlas-um that consume any 3d party services
the specific APIs are mocked or stubbed, and not physically involved.

### Integration tests

During integration testing (using
[Gherkin](https://cucumber.io/docs/gherkin/reference/) feature files),
the USM is mocked using
[MockServer](https://www.mock-server.com). atlas-um is configured to
use the MockServer instance by running both in [Docker
Compose](https://docs.docker.com/compose/). The make command
`docker/test` is provided as a convenient way to do this.

### Async tasks
Async tasks are implemented via the [Celery](https://docs.celeryproject.org/en/stable/).
`atlas_um.tasks` module is intended to use only for thin shortcuts.
The actual business logic should be implemented as logic services and 
placed inside the respective packages.   
Celery broker and backend are configured with the following
environment variables: CELERY_BROKER_URL, CELERY_RESULT_BACKEND

### Code style
[Flake8](https://github.com/PyCQA/flake8) linter is being used to
keep the code style in a good and PEP8 compliant shape. 

[Black](https://github.com/psf/black) is being used for code autoformatting. 
To format *.py files inside the project tree, use:
`make docker/format`. Make sure to run it before the commit.

### Security check
[Pip-audit](https://pypi.org/project/pip-audit/) if being used for the 
security check of the project dependencies. 

[Bandit](https://github.com/PyCQA/bandit) is being used for the 
security check of the project code.

### Frontend

Frontend styling is implemented with [Formantic UI](https://fomantic-ui.com/) framework.
All raw CSS assets are located under the `assets` directory.
For local development [Node.js](https://nodejs.org/uk/) is required.  
Install all needed dependencies with `npm install` inside the `assets` folder.
Use the 
```shell
$ cd node_modules/fomantic-ui
$ npx gulp install
$ npx gulp build && npx gulp watch
```
during development to produce the ready css.


## Postgres database integration

Uses [sqitch](http://sqitch.org/) to manage database migrations using
plain SQL commands, wrapped in several layers of integration:

1. The sqitch tool itself: https://sqitch.org/docs/manual/sqitch/
2. An entrypoint script:
[migration_entrypoint.sh](migrations/migration_entrypoint.sh)
3. A minimal Dockerfile that installs sqitch, its Postgres dependency,
and the entrypoint script:
[Dockerfile](./migrations/Dockerfile)
4. A Docker Compose service that defaults to deploying all migrations
to the Postgres test service container:
[pg_m](./docker-compose.yml)
5. A Makefile entry that builds and runs the `pg_m` service:
[docker/pg/migrate](./Makefile)

When performing local development, both forward and reverse migrations
are integrated into the test suite by running `make docker/test`.

The [entrypoint
script](migrations/migration_entrypoint.sh) is intended
to support the use case of deploying database migrations to a running
database. As described in the entrypoint comments and usage message,
the database to deploy to may be specified by providing a [database
connection URI](https://metacpan.org/pod/URI%3A%3Adb) as an argument
(e.g. `migration_entrypoint.sh -u db:pg://user:pass@host:port/name`)
or by specifying the same env vars as are used by the app to configure
Postgres database access: `PGDB_HOST`, `PGDB_NAME`, `PGDB_PASS`,
`PGDB_PORT`, and `PGDB_USER`. The entrypoint script defaults to
running `sqitch deploy`, but the command can be overridden with the
`-c` option. Additional arguments to sqitch may be passed after after
the argument seperator `--`.

The [migration Dockerfile](./migrations/Dockerfile) just
installs the entrypoint script and its dependencies, and may be
controlled in the same manner, either by passing command arguments via
`docker run` to the entrypoint, or by specifying the same env vars.

The `pg_m` Docker Compose service is intended to be used for service
coordination during automated testing and local development. It should
be possible to use it to deploy migrations to external Postgres
servers, but it has not been designed for that use case. Users wanting
to use containers to manage migrations of live databases are directed
to the [migration Dockerfile](./migrations/Dockerfile).

## DB data initialization
Default DB data initialization is implemented via the cli command.
Run `flask data_init`  inside the proper env`s container to list 
available options.
TODO: define devops strategy for this.

## Auth0

### In a server environment

The altas-um backend authenticates with the [Auth0 Management
API](https://auth0.com/docs/api/management/v2) using the [Client
Credentials
Flow](https://auth0.com/docs/flows/concepts/client-credentials)
defined in [OAuth 2.0 RFC 6749, section
4.4](https://tools.ietf.org/html/rfc6749#section-4.4). This process is
initialized using the environment variables AUTH0_DOMAIN,
AUTH0_CLIENT_ID, and AUTH0_CLIENT_SECRET.

### In local development

If you wish to run atlas-um locally against a real Auth0 tenant,
configure it in the same way as on an independent server, using the
environment variables AUTH0_DOMAIN, AUTH0_CLIENT_ID, and
AUTH0_CLIENT_SECRET. *Warning:* Configuring atlas-um with the
necessary secrets for an Auth0 tenant means that atlas-um can make
changes to the accounts in that tenant!

Alternately, it may be more convenient to force atlas-um to use an
existing access token that is already procured and available within
the local environment. This can be accomplished by setting the
environment variable AUTH0_ACCESS_TOKEN to a valid, current access
token. Note that this skips the client credential flow, and therefore
does not require AUTH0_CLIENT_ID or AUTH0_CLIENT_SECRET to be set, but
it still requires setting AUTH0_DOMAIN. Be advised that in this case,
the user assumes responsibility for renewing the access token, as
atlas-um will not attempt to do so.


## Claims

The means by which atlas-um provides account authorization to resource
servers in the DnA group is via claims made in the bearer tokens it
issues to user agents. A claim is an assertion about a user
account. Claims may indicate, for instance, what label a user works
for, what artist a user should see data for, or what role the user
should be granted on a particular resource server. Claims are
discussed the controlling document for JWTs: [RFC
7519](https://tools.ietf.org/html/rfc7519). nothing in this document
is intended to contradict RFC 7519, and for the purposes of developers
of applications interacting with atlas-um RFC 7519 is intended to be
entirely sufficient for understanding the use of claims in atlas-um.

When considering how atlas-um understands claims internally, most
claims can be thought of relationally as the triple:

    ( <account identity>, <claim name>, <claim value> )

The account identity uniquely identifies a user account
within the DnA group, and in many cases uniquely identifies a user of
DnA services. For the purposes of atlas-um, account identity is
generally equated with the SME-DnA subject assigned to user accounts
by atlas-um.

The claim name is the term used in atlas-um to identify a type of
claim that may be made against accounts. For instance, `iat`, `exp`,
and `iss` are claim names in the atlas-um parlance that are specific
parts of the JWT spec. For claim names not specified by an RFC or
other external document, claim names are uniquely identified by a
URL. For instance, when associating a user account with a label for
use in Decibel services, the claim name may be uniquely identified by
the URL `https://ref.atlas.stream/claims/decibel/label`. The URL need
not be resolvable via DNS, but is given as a URL for namespacing
purposes, or what RFC 7519 refers to as “collision resistance.”

The claim value is the value conveyed by the claim for the given
account identity in the area of the claim name. For instance, the
claim value for an `exp` claim is a Unix epoch timestamp representing
the time at which the token expires, per the JWT spec. An example of
an SME-DnA-specific claim value might be that within Decibel services
some user accounts are ssigned a role of MctUser, in which case the
claim value might be the string `"MctUser"`. Similarly, a claim value
representing that an account should be allowed admin privileges on a
given service may be the value `True`. Claim values may be any valid
JSON value, including a scalar, an object, or an array. The claim
value itself does not necessarily indicate one particular conceptual
value: a claim value of the JSON number `1601586854` may represent a
count of a little more than 1.6 billion of something, or it may be a
Unix epoch timestamp, or it may be a cryptographic nonce; the context
for how to interpret a claim value is provided by the claim name with
which it’s associated.

Putting that all back together, a claim in atlas-um is composed of: an
account identity, represented by a unique account identifier in the
SME-DnA space, e.g. `sme-dna|cf339fa4d5a4b4883597ade0d0fc9be3`; a
claim name, represented by a URL,
e.g. `https://ref.atlas.stream/claims/rti/artists`; and a claim value,
represented by JSON value, e.g. `[{"gras_id": "4355"}, {"gras_id":
"1748784"}]`.


## Code updates

Before you push code and create a PR, run the following command:

`make pre-commit`


## Static files

Static files in docker image are ready for the final deployment to
external CDN/Storage. Copy them from `atlas_um/static` and then use 
the following environment variable to override the static host on
specific env `STATIC_URL`, e.g:  `STATIC_URL=https://static.example.com`