# playwrightTests

Run Playwright-based end-to-end tests from the `playwright-tests` repository inside a Docker Compose provided test
runner. The step:

- Checks out the `playwright-tests` repo (supports regular branches and pull requests)
- Injects framework / runtime / config environment variables plus any custom ones you pass
- Assumes an AWS role to run (via `withAWS`)
- Executes the test
  runner: `docker compose pull framework-runner && docker compose run --rm framework-runner`
- Archives the Playwright HTML report and publishes it as a Jenkins HTML report named "Playwright Report"
- Automatically sends Slack notifications to `#e2e-test-results` when tests fail (regression) or recover (fixed)

## Parameters

| Name                      | Description                                                                                                                                              | Type     | Default           | Required |
|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------------------|----------|
| tags                      | Playwright tag expression (passed through as `TAGS`) used to filter which tests run.                                                                     | `String` | n/a               | yes      |
| frameworkEnvironment      | Framework environment (The test or production code of the framework itself).                                                                             | `String` | `prod`            | no       |
| runtimeEnvironment        | Runtime execution environment, defaults to `lambda` can be set to `ecs` for long running tests.                                                          | `String` | `lambda`          | no       |
| configEnvironment         | The environment of the application under test. Can be `qa` or `prod`                                                                                     | `String` | `qa`              | no       |
| branchName                | Git branch (supports multibranch style e.g. `*/master`). If a PR pattern `PR-123` is detected it fetches that PR head.                                   | `String` | `*/master`        | no       |
| frameworkRunnerTag        | ECR tag of the framework-runner Docker image to use for test execution.                                                                                  | `String` | `latest`          | no       |
| githubUser                | GitHub org / user that owns the `playwright-tests` repository.                                                                                           | `String` | `theorchard`      | no       |
| reportSuffix              | Suffix for report name and subdirectory. Allows multiple test runs in the same pipeline with unique reports.                                             | `String` | empty (`''`)      | no       |
| slackNotificationChannels | Additional Slack channels to notify (in addition to the default `#e2e-test-results`).                                                         | `List`   | empty list (`[]`) | no       |
| envVars                   | Additional key/value environment variables to append (merged into container env).                                                                        | `Map`    | empty map (`[:]`) | no       |
| pnpmSetupScriptName       | Standalone script name that is needed to be executed before main suite (if any). Runs once in a container on jenkins before the test runs are triggered. | `String` | empty (`''`)      | no       |

## Injected Environment Variables

These are always provided to the container (plus anything you supply via `envVars`):

- FRAMEWORK_ENV
- RUNTIME_ENV
- CONFIG_ENV
- FRAMEWORK_RUNNER_TAG
- TAGS
- REPORT_SUFFIX (only if `reportSuffix` parameter is provided)
- ADDITIONAL_ENV_VARS (JSON array of custom environment variables from `envVars` parameter)
- PACKAGECLOUD_TOKEN (from Jenkins credentials `packagecloud_repo_token`)
- GITHUB_TOKEN (from Jenkins credentials `github_packages_token`)

## AWS Role

The step calls:

```
withAWS(role: "${env.FRAMEWORK_ENV}-playwright-tests-role", roleAccount: '437795906767', ...)
```

Ensure a role matching the computed name (e.g. `prod-playwright-tests-role`, `qa-playwright-tests-role`) exists and is
assumable by the Jenkins AWS credentials.

## Reports

- Published HTML report: directory `report`, file `index.html`, name "Playwright Report" (or "Playwright Report - {reportSuffix}" if suffix provided)
- Each test run is isolated in its own subdirectory: `playwright-tests/{reportSuffix}` (defaults to `playwright-tests/default`)

## Usage

### Basic

```groovy
stage('Playwright E2E') {
  steps {
    playwrightTests(tags: '@smoke')
  }
}
```

### With Custom Environment Variables and Framework Runner Tag

```groovy
stage('Playwright E2E') {
  steps {
    playwrightTests(
      tags: '@checkout',
      configEnvironment: 'staging',
      frameworkRunnerTag: 'v2.1.0',
      envVars: [
        API_BASE_URL: 'https://staging.api.internal',
        FEATURE_FLAG_CART: 'true'
      ]
    )
  }
}
```

### Multiple Test Runs in Same Pipeline

When running multiple `playwrightTests` steps in the same pipeline, use the `reportSuffix` parameter to ensure each run has its own workspace and generates a unique report. This prevents conflicts and allows you to view all test results separately.

```groovy
stage('Playwright Tests') {
  parallel {
    stage('QA Environment') {
      steps {
        playwrightTests(
          tags: '@smoke',
          configEnvironment: 'qa',
          reportSuffix: 'qa'
        )
      }
    }
    stage('Prod Environment') {
      steps {
        playwrightTests(
          tags: '@smoke',
          configEnvironment: 'prod',
          reportSuffix: 'prod'
        )
      }
    }
  }
}
```

This will create:
- Two separate workspaces: `playwright-tests/qa` and `playwright-tests/prod`
- Two separate HTML reports: "Playwright Report - qa" and "Playwright Report - prod"
- The `REPORT_SUFFIX` environment variable passed to each test run

### Pull Request Build

If your multibranch pipeline names PR branches like `PR-123`, just supply `branchName: 'PR-123'` (or rely on the
detected `env.GIT_BRANCH`). The library converts it to the correct fetch refspec internally.

```groovy
playwrightTests(tags: '@smoke', branchName: 'PR-123')
```

## Slack Notifications

The step automatically sends Slack notifications in two scenarios:

- **Regression**: When tests fail after previously passing (or on first run)
- **Fixed**: When tests pass after previously failing

By default, notifications are sent to `#e2e-test-results`. You can specify additional channels using the `slackNotificationChannels` parameter - all channels will receive notifications.

Notifications include the job name, build number, and a link to the build. If `reportSuffix` is provided, it's included in the notification message to identify which test run failed or recovered.

### Example with Additional Channels

```groovy
stage('Playwright E2E') {
  steps {
    playwrightTests(
      tags: '@smoke',
      slackNotificationChannels: ['#team-alerts', '#qa-notifications']
    )
  }
}
```

## Notes

- Provide only valid tag expressions supported by your Playwright tagging strategy.
- **Use `reportSuffix` whenever running more than one `playwrightTests` step in the same pipeline** to avoid workspace collisions and ensure unique reports.
- `envVars` must be a Map. An empty map literal is `[:]` (not `[]`, which would be a List).
- `envVars` are converted to a JSON array and passed as `ADDITIONAL_ENV_VARS` environment variable.
- `frameworkRunnerTag` allows you to specify which version of the framework-runner Docker image to use from ECR.
- The underlying repository is expected at: `git@github.com:${githubUser}/playwright-tests.git`.
- Slack notifications are sent automatically and do not require any configuration in consuming Jenkinsfiles.

## envVars Reference

These variables can be passed via the `envVars` parameter to configure framework and test behaviour.

### Test Execution

| Variable | Type | Default | Description |
|---|---|---|---|
| `TRACE` | `"on"` \| `"off"` \| `"on-first-retry"` \| `"on-all-retries"` | `"on-all-retries"` | Playwright trace recording mode. Set to `"on"` to capture traces for all tests — useful when debugging intermittent failures. |
| `USE_DYNAMIC_THROTTLING` | `"true"` \| `"false"` | `"false"` | When `"true"`, limits concurrent feature file execution instead of running all files in parallel. Use for large suites that hit Lambda/ECS resource limits. |
| `MAX_CONCURRENT_FEATURES` | integer string | — | Hard cap on concurrent feature files. Only applies when `USE_DYNAMIC_THROTTLING=true`. If omitted, concurrency is auto-calculated as ~40% of total file count (minimum 1). |

### Application Under Test

| Variable | Type | Default | Description |
|---|---|---|---|
| `BASE_URL_OVERRIDE_<APPNAME>` | URL string | — | Override the base URL for a specific app. Replace `<APPNAME>` with the uppercased app name. Supported values: `FANSIFTER`, `INSIGHTS`, `WORKSTATION`, `OA`, `DOCUMENTS`, `ABACUS`, `COLLABORATORS`, `CONTENT`, `DISTRIBUTION`, `PUBLISHING`, `MONEYHUB`, `SETTINGS`, `AUTH`. |
| `TEST_WISE` | `"false"` | enabled | Set to `"false"` to skip real TransferWise payment processing in collaborator payment tests. Use when testing payment UI/logic without executing actual financial transactions. |

### ECS Runtime

| Variable | Type | Default | Description |
|---|---|---|---|
| `TASK_REVISION` | integer string | latest | Pin the ECS task definition to a specific revision (e.g. `"42"` → `prod-playwright-tests:42`). Only applies when `runtimeEnvironment: 'ecs'`. |

### Diagnostics

| Variable | Type | Default | Description |
|---|---|---|---|
| `DEBUG` | `"true"` | disabled | Enables `[DEBUG]`-prefixed log output throughout the framework and test code. Useful for diagnosing test setup issues. |

### Examples

```groovy
// Capture full traces for a flaky suite
playwrightTests(
  tags: '@flaky_suite',
  envVars: [TRACE: 'on']
)

// Throttle concurrency for a large regression suite
playwrightTests(
  tags: '@regression',
  runtimeEnvironment: 'ecs',
  envVars: [USE_DYNAMIC_THROTTLING: 'true', MAX_CONCURRENT_FEATURES: '10']
)

// Skip real TransferWise transactions
playwrightTests(
  tags: '@collaborator_payments',
  envVars: [TEST_WISE: 'false']
)

// Override the OA base URL
playwrightTests(
  tags: '@oa_smoke',
  envVars: [BASE_URL_OVERRIDE_OA: 'https://custom-oa.internal']
)
```