# moneyhub Lambda functions for customer accounting. ## Requirements * [uv](https://docs.astral.sh/uv/) - manages the virtual environment and the correct Python version for each lambda automatically (see the `.python-version` in each lambda's directory) * [Docker](https://www.docker.com/) ## Setup * Install [docker](https://docs.docker.com/get-docker/) * Install [uv](https://docs.astral.sh/uv/getting-started/installation/) * Clone this repository * Navigate to the lambda you want to work on in `lambda/` ## Installing Make a copy of the environment file and fill it with your details: ```bash cp .env.shadow .env ``` There is no separate environment setup step: uv creates and syncs the virtual environment (in `env/`) automatically the first time you run any `make` target or `uv run` command. To create/sync it explicitly: ```bash uv sync ``` Note: you need to be on the VPN the first time (and whenever the lock file changes) - dependencies come from the private package index. ## Authorization This lambda connects to `ows-moneyhub`, which enforces Authorization in both QA and PROD environments. By default, our lambda client connects to `ows-moneyhub` in QA. Optionally, you can connect to `ows-moneyhub` in the `Dev` environment which does not enforce authorization and skip the next steps. See the below section on _"Connecting to Local Microservices"_. If you're connecting to QA you will need an authorization token, which can either be your own or be the M2M token from the secrets manager. ### Using your own token Using your own token can be done by logging in to one of the QA frontends and copying it from the request headers sent there. It will be the `authorization` header and you copy what is after the "Bearer" text: ```json { "authorization": "Bearer eyJhbGciOiJSUzI1NiIsInR5cCI..." } ``` Place this in the `OWS_CLIENT_TOKEN` environment variable in your .env file and all requests should use this token. ### Using the M2M token To use the M2M token you will first need to have access to the following secrets in the Secrets Manager (in the dev AWS account): * `dev/lambda-jwt-refresh/jwt_token` * `dev/lambda-jwt-refresh/jwt_token_expiration` You can [request access via DevOps](https://www.notion.so/DevOps-End-User-Support-Process-fae6dcaca43e48e28bf9f601bdaa7722) using [this ticket](https://theorchard.atlassian.net/browse/SYS-21157) if you do not already have access. Before running the lambda, ensure you have a valid AWS token for the dev environment by running the `awsume` tool: ```shell awsume dev ``` If everything is configured correctly you should see the following when running the lambda: ``` Fetching M2M token from cache ... Fetching M2M token from secrets manager ... Found credentials in environment variables. ``` ## Connecting to Local Microservices By default, when running any of the lambdas locally, you will connect to the respective microservices in their QA environment. If you would like to connect to a local microservice, you can add the following line to your `.env` file: ```python OWSREQUEST_SERVICE_MAP='{"ows-moneyhub": "http://localhost:5000"}' ``` ## Feature Flags _Note: Currently only the `generate_attachments` lambda checks feature flags._ The lambdas may check specific feature flags in Split.io to differentiate between logic. If you have a need to mock a specific flag for development/testing you will need to create a `.split` file in the lambda's directory and add/uncomment this line in your `.env` file: ``` SPLIT_FILE_PATH='./.split' ``` The format of the file is one flag per line separated by a space and either `on` or `off`: ``` moneyhub_very_cool_feature on moneyhub_less_cool_feature off moneyhub_other_feature on ``` ## Running The lambda function can be run locally as a python function or as a container. Run the python function with a mock event: ```shell make run ``` Run the dockerized function by first bringing up the container: ```shell make local_listener ``` This container will act similar to a web server by listening on a port for the event data to be POSTed. POST example data to the lambda: ```shell make local_event ``` ### Start Container ```shell docker-compose up --build -d ``` The lambda service is named after the lambda (e.g. `lambda-custom-reports`, `lambda-generate-attachments`) - see the `docker-compose.yaml` in the lambda's directory. ### Execute Function Use the HTTP client of your choice. The body of the request is the `event` passed into the function. ```shell curl --request POST \ --url http://localhost:9000/2015-03-31/functions/function/invocations \ --header 'Content-Type: application/json' \ --data $(cat tests/sample_event.json) ``` ### Load Changes The `src` directory and `config.py` are mounted into the container via the `docker-compose.yaml` configuration. This means you don't need to re-build the entire container to see changes. **However** the container will need to be restarted. This process should be as quick as restarting uwsgi when making changes to a web application. ```shell docker-compose restart ``` :zap: You should add this command to a post-write hook in your preferred editor to automatically restart. ## Linting and Testing ### Run Container ```shell docker-compose up --build lint-and-test ``` ### Set pytest Options Sometimes you may want to customize how tests run using [pytest options](https://docs.pytest.org/en/stable/usage.html) when doing development locally, only running a subset of tests using `-k` for example. Simply edit `TEST_ARGS` value in `docker-compose.yaml`, **without** quotes, to set the options you need. :warning: Commited changes will modify how the build pipeline runs tests ### Set pytest-cov Options By default test coverage is calculated for files in `./src` and displayed in a table after tests run. * If you want to see a detailed output, edit `COV_REPORT` value in `docker-compose.yaml` to be `html` and visit `http://localhost:8000/` * If you want to turn off coverage report generation, edit `COV_REPORT` value in `docker-compose.yaml` to be `off` :warning: Commited changes will modify how the build pipeline runs tests, **do not** commit `COV_REPORT=html` ### Skip Linting When developing locally, you might not want to worry about linting when working to make tests pass. Simple edit the `SKIP_LINT` value in `docker-compose.yaml` to be `1`. :warning: Careful not to commit changes unless you **really** want to not run linting on the build pipeline ### Capture Exit Code Unless the container itself crashes, or is killed, the exit code will be `0`. When the exit code of `lint-and-test.sh` script matters, a pull-request builder for example, run tests with the following flags. ```shell docker-compose up --exit-code-from lint-and-test --abort-on-container-exit --build lint-and-test ``` ### Datadog Configuration In order to get traces and metrics (in addition to logs) from the [Datadog Lambda Library](https://docs.datadoghq.com/serverless/installation/python/?tab=containerimage), you'll need to have the following environment variables: DD_ENV (dev, qa, or prod) DD_SERVICE (the name of your service) DD_API_KEY_SECRET_ARN or DD_API_KEY (depending on how you are accessing the api key from secrets manager) DD_LAMBDA_HANDLER (your lambda's original handler, e.g. "src.app.handler") DD_TRACE_ENABLED (set to true) These can be set via terraform, for example [here](https://github.com/theorchard/terraform-infra/blob/master/qa/lambda-sound-recordings/lambda_sr_add_version.tf#L84-L91). ## Testing Run unit and functional tests: ```shell make test ``` Run linting ([ruff](https://docs.astral.sh/ruff/) - checks and formatting): ```shell make lint ``` Auto-fix lint issues and reformat: ```shell make format ``` Run type checks (mypy): ```shell make types ``` ## Managing Dependencies Dependencies are declared in each lambda's `pyproject.toml` and locked in `uv.lock`. After changing `pyproject.toml`, regenerate the lock file (requires VPN): ```shell make lock ``` To regenerate it inside a Linux container instead (matches what the deployed image resolves): ```shell docker-compose run --rm lockfile ``` Both `pyproject.toml` and `uv.lock` must be committed together.