# GraphQL Performance Testing with k6 This project is designed to test the performance of GraphQL APIs and frontend page loads using the k6 performance testing tool. It includes scripts for backend GraphQL requests and frontend page visits, with metrics tracking for trends, failures, and successes. ## Table of Contents - [Project Overview](#project-overview) - [Technologies Used](#technologies-used) - [Setup](#setup) - [Environment Variables](#environment-variables) - [Usage](#usage) ## Project Overview The project includes: - Backend GraphQL API performance testing. - Frontend page load testing with Puppeteer. - Metrics tracking for request durations, failures, and successes. ## Technologies Used - **Languages**: JavaScript, TypeScript - **Tools**: k6, Puppeteer - **Package Manager**: yarn ## Setup 1. Clone the repository 2. Install dependencies: ```bash yarn install ``` 3. Create a `.env` file in the root directory and set the required environment variables from .env.shadow. ## Environment Variables You will need all environment variables in the .env.shadow file to run the tests. See the table below for a description of each variable. | Variable Name | Description | |--------------------------------|------------------------------------------------------------------------------------------| | `GRAPHQL_URL` | The base URL of the GraphQL API to be tested (Likely via grass). | | `APP_URL` | The base URL of the application to be tested (e.g. `https://example.com`). | | `BACKEND_SERVICE` | The base URL of the microservice to be tested (e.g. `https://example.com`). | | `K6_BACKEND_STAGE_1_DURATION` | Duration in seconds of the first backend test stage (e.g. `10s`). | | `K6_BACKEND_STAGE_1_VUS` | Number of virtual users for the first backend test stage to ramp up to. | | `K6_BACKEND_STAGE_2_DURATION` | Duration in seconds of the second backend test stage (e.g. `10s`). | | `K6_BACKEND_STAGE_2_VUS` | Number of virtual users for the second backend test stage to ramp up to. | | `K6_FRONTEND_VUS` | Number of virtual users for frontend (browser) tests. | | `K6_FRONTEND_START_TIME` | Start time offset for frontend tests (e.g. `00s`). | | `K6_FRONTEND_DURATION` | Duration of the frontend test stage (e.g. `20s`). | | `K6_WEB_DASHBOARD` | Set to `true` to enable the k6 web dashboard during test execution. | | `K6_WEB_DASHBOARD_EXPORT` | Path to export the k6 web dashboard HTML report (e.g., `/reports/html-report.html`). | | `K6_METRIC_THRESHOLDS` | A string array of JSON objects detailing which metrics to assert after a run (see below) | ## Usage ### Docker The easiest way to run the k6 tests is with Docker. This requires no installation of k6 on your local machine. Run the tests with the below command passing in the path to your test file. For example: ```bash make build_and_run_docker_k6_test TEST_FILE=abacus/contract-search-and-visit-test ``` ### User Setup The users are set up in an AWS secret that will be fetched at runtime. You can fetch the file using the make command. ```bash make fetch_secret_files ``` You will need to add your users following the format found there. There is a template file `user-data.shadow.json` for further reference. ### GraphQL Queries The GraphQl Queries are fetched dynamically from the source repositories at runtime. You can fetch the queries using the make command. ```bash make fetch_gql_queries ``` To add more queries, add to src/frameworkScripts/queryFetcher/gqlQueryFileLocations.json. The queries are fetched from the source repositories at runtime to ensure they are up to date. Currently, the queries in .gql and .ts files are supported but you may find you need to update `fetchGqlQueries.ts` to support your query format. You can also add queries directly to `src/k6/graphqlQueries/app/` if necessary although this is considered an anti-pattern. If you do this you will also need to ensure you add an `index.ts` file to the dir to export the queries. ### API load testing API load testing is implemented with ease in mind to support most of the ows-microservices without need to write any code, just a json file with all the configuration and you are ready to test. The API load testing can be performed for any available microservice using rest api (for graphql microservices see above) by adding a new file with test configuration under `src/variables/endpoints/` like `ows-vectororder` ```json { "baseUrl": "qa-ows-vectororder.theorchard.io", "scenarios": { "eligibility": { "vus": 1, "stages": [ { "duration": "1s", "target": 1 }, { "duration": "2s", "target": 1 } ], "gracefulStop" : "2s", "endpoints": [ { "method": "GET", "path": "/jobs/506581949/eligibility" }, { "method": "GET", "path": "/jobs/508480984/eligibility" }, { "method": "GET", "path": "/jobs/484137613/eligibility" } ] } } } ``` OR ```json { "baseUrl": "qa-ows-vectororder.theorchard.io", "scenarios": { "eligibility_1": { "vus": 1, "stages": [ { "duration": "1s", "target": 1 }, { "duration": "2s", "target": 1 } ], "gracefulStop" : "2s", "endpoints": [ { "method": "GET", "path": "/jobs/506581949/eligibility" } ] }, "eligibility_2": { "vus": 1, "stages": [ { "duration": "1s", "target": 1 }, { "duration": "2s", "target": 1 } ], "gracefulStop" : "2s", "endpoints": [ { "method": "GET", "path": "/jobs/508480984/eligibility" }, { "method": "GET", "path": "/jobs/484137613/eligibility" } ] } } } ``` You may also define a specific user for each scenario by including the username key in the JSON configuration file, as shown in the example below. ``` { "baseUrl": "qa-ows-vectororder.theorchard.io", "scenarios": { "eligibility_1": { "username": "userA@theorchard.com" ... }, "eligibility_2": { "username": "userB@theorchard.com", ... } } } ``` You may also define a specific slack channel to get a small summary of the execution by including the slackChannel key in the JSON configuration file, as shown in the example below. If no key added, functionality defaults to qa automation team internal channel. ``` { "baseUrl": "qa-ows-vectororder.theorchard.io", "slackChannel": "#your-channel" can add more than one separated with comma "scenarios": { "eligibility_1": { "username": "userA@theorchard.com" ... }, "eligibility_2": { "username": "userB@theorchard.com", ... } } } ``` Slack message example: ``` Aggregated results for ows-vectororder: Request Count: 23334 Failure Count: 248 Average Response Time: 4.161 ``` ### Threshold Assertion The `K6_METRIC_THRESHOLDS` environment variable is a JSON string array that defines the thresholds for various metrics. This will check against the output file reports/summary.json after the test run. Time values are in milliseconds. The framework will assert that metrics are less than or equal to the specified thresholds. If any threshold is not met, the test will fail. ```json [ {"key":"frontend_journey_duration","metric":"avg","threshold":2000} ] ``` ### Typescript Tests are written in typescript and compiled at runtime using webpack. You will find the `tsconfig.json` file for the test code in src/k6. If you need to add additional dir's to the suite, ensure they are also added to webpack.config.js. Webpack will bundle the test file given by the `TEST_FILE` env var and output a js bundle to `./k6Ready` which will be run by k6. Framework scripts live in src/frameworkScripts. Their `tsconfig.json` is found in that dir. They are compiled when the container is built and are run via node.