# Grass ## Table of Contents * [Overview](#overview) + [Authorization](#authorization) + [Access Methods (active ones)](#access-methods-active-ones) + [How to use Route based session tokens](#how-to-use-route-based-session-tokens) - [Route's yaml file](#routes-yaml-file) - [Route structure](#route-structure) - [Sample yaml files](#sample-yaml-files) - [Adding allow_session_access_for_routes to microservice](#adding-allow_session_access_for_routes-to-microservice) - [ALW Roles](#alw-roles) - [OA Resources](#oa-resources) + [Logging](#logging) + [Questions](#questions) * [Getting Started](#getting-started) + [Requirements](#requirements) + [Installation](#installation) + [Launch the server](#launch-the-server) + [Running formatters / linters](#running-formatters--linters) + [Running tests](#running-tests) + [Running API tests](#running-api-tests) + [Specifying custom service resolution rules](#specifying-custom-service-resolution-rules) + [Session Token](#session-token) + [Additional Reading](#additional-reading) ## Overview Grass is our authorization API proxy. It sits between our frontend applications and our fleet of microservices. GRASS takes public incoming requests from frontend apps, verifies the user has already authenticated as a specific user, and that the user's application & role is authorized to access a particular microservice. GRASS then adds user metadata to the requests Headers that the downstream service can use to perform any additional authorization checks. See [GRASS - Overview & Dev Setup](https://docs.google.com/presentation/d/1TridG1Z-BQCPEI7nzqhm4Fvym6Q079OsMyEcCR7MfgY) for a detail walkthrough of this. A recorded GRASS workshop video explaining these slides can be viewed [here](https://drive.google.com/open?id=1WWD-OtFOjzKSCdv7ARcrpNaMt4ziQln7). ### Authorization There are today 2 ways a user can be authorized to access a service: * **Routes based Session Tokens** [`(allow_session_access_for_routes)`](https://github.com/theorchard/ows-grass/blob/master/grass/access.py#L305): Recommended. Those short lived tokens (20 minute) are generated by a specific client (OA / ALW). This allows Frontend applications to make calls directly against Grass. To get a session token, the legacy clients must use their "vector access tokens". Currently it checks for ALW user roles and OA user access to resources. Refer section below on how to it works. * **Vectorapi Access Tokens** [`(allow_client)`](https://github.com/theorchard/ows-grass/blob/master/grass/access.py#L191): Not recommended (scheduled for deprecation). Used by legacy systems. When OA / ALW wants to make a call on the behalf of a user, the vectorapi access token is used. All backend calls from legacy system to microservice or microservice to microservice calls should not go via grass. They should be direct calls to microservice. ### Access Methods (active ones) * `allow_session_access_for_routes()` - Route specific access. See how to use section below. * `allow_auth0_token()` - Allow an API endpoint to accept an Auth0 oAuth2 Access Token. It also verifies that Orchard-User-Id is one of the linked accounts. Going forward we want to switch to `allow_jwt_token` when we start assigning profiles for all user types. * `allow_jwt_token()` - Allow an API endpoint to accept Bearer JWT Token in Authorization header. Verifies JWT is valid and has orchard_identity_id. If profile headers are present they are verified that the values match profiles in JWT. This will block any JWT trying to access profiles other than their own. * `allow_auth0_m2m_token()` - Allows an API endpoint to accept an Auth0 M2M application Access Token ### How to use Route based session tokens `allow_session_access_for_routes()` takes a yaml file as mandatory argument which contains all allowed routes and corresponding OA resources and ALW roles that are allowed to access that route. * #### Route's yaml file - All the routing yml files should be in `/grass/rules/` folder. - Filename should be same as the name of the microservice. - Start the yml with `rules` node and all routes should be under it. See samples below - It works by checking each rule to see if it matches current url + http-method + user-roles/user-resource-access. If any one of the route matches it returns true. - Only add routes from your microservice that will be called from frontend via grass. This way we can protect other routes from accidentally being exposed if they will be called only from other microservices. - EndpointRule is flexible to support regex style too, but flask style is preferred for ease of readability. * #### Route structure ``` - path: /url as recieved by grass. methods: [comma seperated list of HTTP methods for this path] groups: alw: [comma seperated alw role ids] oa: [comma seperated OA resource names] ``` - _path_: should be url as recieved by grass and not as it is mentioned in handler. For eg: grass receives `/account/subaccounts` but in ows-account handler we have `@app.route('/subaccounts', methods=['GET'])` so path should be `path: /account/subaccounts` - _operators for path_: `` - will match 1+ numeric value `` - will match 1+ character from `a-z, A-Z, 0-9, -` `<*>` - will match any number of any characters or none - _operators for roles / resources_: Instead of comma separated roles / resources you can provide `'*'` if it is allowed for every role. * #### Sample yaml files ``` rules: # Catalog, Analytics and Master Rights: distributor check allowed. - path: /account/distributor methods: [HEAD] groups: alw: [1, 3, 6] oa: ['*'] # Admin: Allow all. - path: <*> methods: ['*'] groups: alw: [4] oa: ['*'] ``` ``` rules: # Allow all to get conflicts types, as they are used on homepage. - path: /conflict-manager/conflicts/new methods: [GET] groups: alw: [1, 2, 3, 5, 6] oa: ['*'] # Master Rights: Get on all 3 conflicts endpoints and post actions - path: /conflict-manager/action<*> methods: [POST] groups: alw: [1, 6] oa: ['*'] - path: /conflict-manager/conflicts/<*> methods: [GET] groups: alw: [1, 6] oa: ['*'] # Admin: Allow all. - path: <*> methods: ['*'] groups: alw: [4] oa: ['*'] ``` ``` rules: # Allow access to pricing/admin routes for users having only Pricing Resource access. - path: /pricing/admin/<*> methods: [POST, PUT, DELETE] groups: oa: [pricing] ``` * #### Adding allow_session_access_for_routes to microservice ``` access=[ access.allow_session_access_for_routes( RULES_YML_PATH + 'ows-xxxx.yml'), ``` * #### ALW Roles - 1 Catalog – view and manage your catalog at The Orchard - 2 Marketing – permission to view, install, and use Apps in the Orchard Marketing - 3 Analytics – view everything under the analytics tab, including sales data for any release in your catalogue - 5 Accounting – view everything under the accounting tab, including all financial information and your Travelex account - 6 Manage Rights – edit track metadata required to collect performance (neighbouring) rights royalties and mediate rights conflicts - 4 Administrator * #### OA Resources - Resources are present in orchadmin_resources table. ``` SELECT * FROM art_relations.orchadmin_resources; ``` - allow_session_access_for_routes performs [privilege check for OA users against the resources](https://github.com/theorchard/ows-grass/blob/master/grass/logic/endpoint_rules.py#L192) listed in the rules yaml file. ### Logging * Grass runs a set of Tornados in Elastic Beanstalk. Nginx functions as a proxy on the parent Elastic Beanstalk instance. We ship parent instance access logs to Cloudwatch based on config files in the .ebextensions directory. * To access Cloudwatch logs for Grass, log into AWS and navigate to Cloudwatch -> Logs. You should see access logs for QA and Prod called: `qa-ows-grass-webrequests` and `prod-ows-grass-webrequests`, respectively. ### Questions * _Did Grass authorize my request?_
When Grass authorizes a request, it sets a `Grass-Authorized` header in the response as `Yes`. If the request is not authorized, the error response will be `403: Grass does not authorize this request. Check credentials.` and `Grass-Authorized` will not be set. So if you see a 403 or a 401, to ensure this is coming from Grass, verify the `Grass-Authorized` header is not set. * _Why is my request redirected to the wrong service?_
This issue can happen when the root of the route is shared with another service defined before. Order of registration in services.py takes priority. For instance: if `/auth` is first, and points to `ows-users/auth`, and later you add `/authentic`, which points to `ows-awesome`, `/authentic` will have to be defined before `/auth`. Otherwise requests to `/authentic/foobar` will be redirected to `ows-users/auth/entic/foobar`. * _Why can't Grass handle file uploads?_
Microservices should only deal with references to files. For instance: if you upload a file, the frontend can write the file into S3 automatically, and the file path can be used in the request to grass all the way to the microservice that actually needs it. [See full answer](/theorchard/ows-grass/issues/188). ## Getting Started See below for instructions on setting up a local version of GRASS that can proxy to either QA microservices or locally running services (pay particular attention to the [Specifying custom service resolution rules](#specifying-custom-service-resolution-rules) section). Skip to the 33 minute mark in the [GRASS Workshop video](https://drive.google.com/open?id=1WWD-OtFOjzKSCdv7ARcrpNaMt4ziQln7) for a walkthrough of these steps. ### Installation Before starting make sure you have python3.11 installed. An easy way to manage python versions is with pyenv. ```zsh $ brew update $ brew install pyenv $ pyenv install 3.11.2 $ pyenv local 3.11.2 ``` #### Get the code ```bash git clone https://github.com/theorchard/ows-grass.git cd ows-grass git remote rename origin upstream git remote add origin https://github.com//ows-grass.git git fetch origin ``` #### Create .env file Make a copy of the sample environment file ```bash cp .env.sample .env ``` Fill out values in the .env file for `DB_URL`. If the `Environment` environment is not explicitly set to `qa` or `prod`, the .env file will be read on server start to hyrdate environment variables Optionally, if you have a local redis store and want generated grass session tokens to live between server restarts, fill out the values for `REDIS_HOST` & `REDIS_PORT`. #### Set up a virtual Python environment (local development) When you have the local copy, you need to create a running environment. Run: ```bash make dev ``` ### Launch the server Make sure you are in the virtual environment. You should see `(env) $` prefixed in your prompt. ```bash make dev ``` ### Running tests We use [Pytest](http://pytest.org/) to run our tests. ```bash make test_unit ``` ### Running formatters / linters ```bash make fmt lint ``` ### Running API tests We use [Tavern](https://github.com/taverntesting/tavern) to run our tests. First, configure environment variables in `.env` ```bash QA_DB_USER=automation_qa QA_DB_PASS= QA_DB_HOST=qadb01.qaorch.com QA_DB_DATABASE=art_relations QA_GRASS_HOST=https://api-dev.theorchard.io/auth/session ``` Then run: ```bash make test_int or make docker_test_integration ``` ### Specifying custom service resolution rules While developing it can sometimes be convenient to tell Grass to divert all requests directed at a particular service to an ad hoc instance of that service, e.g., one running locally. For example, we might want to have all services resolve to their deployments on QA except for `royalties`, which we might be running locally. To allow for this sort of workflow, Grass supports "service overrides". Copy the contents of [service_overrides/overrides.py.shadow](service_overrides/overrides.py.shadow) to a file named `overrides.py` in the same folder and register your custom resolvers, following the syntax used in [services.py](grass/services.py). When running in the `dev` environment, Grass will automatically load this file, and the service registrations in it will take precedence over those in the canonical service registry. If you happen to be running Grass as a Docker container, then you can mount your own `overrides.py` inside the container to control how it resolves services, without having to alter the image itself. ### Session Token Session tokens are short lived tokens (20 mins) that are used by our frontend applications to communicate directly through Grass to our different microservices. For development, we've extended the duration of a session token to 30 days. To get a single token: * Go on [QA OA /grasssession](http://oa.qaorch.com/grasssession) endpoint. * Go to the [QA ALW /grasssession](http://workstation.qaorch.com/grasssession) endpoint. For production, you will want to make sure that you have a system capable of fetching a new token every 40 seconds or so. ### Historical GRASS Docmentation This documentation is related to past GRASS decisions. It is not necessarily relevant to the existing code, but is useful to understand past tech decisions. * [Primer](https://docs.google.com/a/theorchard.com/document/d/1dduQXnEyw4lptAg-x1o6fEvpjqBj5O6P9bAWa12QOk0/edit?usp=sharing) * [Tech Design for OA Users on Grass](https://docs.google.com/a/theorchard.com/document/d/1ZVpndOdc6xI7K_60OhAUgXlhBQCK12A6Bvy6WKgHYEY/edit?usp=sharing) * [Tech Design for Communication Security with Grass](https://docs.google.com/a/theorchard.com/document/d/16jIj4SBeInMK2En6mreCTIlOOWgoOoc4VlYJJA7Pt90/edit?usp=sharing) * [Tech Design: Analytics & Grass](https://docs.google.com/a/theorchard.com/document/d/1ym0Bq-tFGQILQe6g6x6oF8f0dS92QD3dnV6aKJF2TvA/edit?usp=sharing) * [Tech Design for Microservice Communication Security](https://docs.google.com/a/theorchard.com/document/d/1eHoI_BddTFMi15yCaHS6KvhSSoTrMEd3WwJINIpgNpM/edit?usp=sharing)