# Playwright Tests

Playwright testing framework with TypeScript, ESLint, and Prettier with BDD support.

## Contents

- [Installation](#installation)
- [Running Tests](#running-tests)
- [Running Tests in Docker](#running-tests-in-docker)
- [Watch & Auto-Regenerate](#watch--auto-regenerate-pnpm-dev)
- [Automatic Scenario ID Tagging](#automatic-scenario-id-tagging)
- [Auto-fix Formatting and Linting](#auto-fix-formatting-and-linting)
- [IDE Setup](#ide-setup)
- [Project Structure & Organization](#project-structure--organization)
- [Adding a New App](#adding-a-new-app-suite-application)
- [Hooks](#hooks)
- [API & Type Modularization](#api--type-modularization)
- [Database Client Usage](#database-client-usage)
- [Onboarding & Best Practices](#onboarding--best-practices)
- [AWS Secrets Management](#aws-secrets-management)

---

## Installation

We use [PNPM](https://pnpm.io/) as the package manager.

To set up this project, follow these steps:

1. Install dependencies:

```bash
pnpm install
```

2. Install Playwright browsers:

```bash
pnpm playwright install
```

## Running Tests

`!!!ATTENTION` You need to have a valid AWS session to access Secrets Manager before running tests. Use AWSUME

### Generate Playwright `.spec` files from BDD Gherkin feature files

```bash
pnpm bddgen
```

### Run all tests

```bash
pnpm playwright test
```

### Run a single test file

```bash
pnpm playwright test .features-gen/features/oa/oa.feature.spec.js
```

### Run tests by tag

```bash
TAGS="@qa_smoke" pnpm playwright test
```

### Using the Makefile (recommended for local runs)

The Makefile wraps `pnpm bddgen` + `pnpm playwright test` into convenient targets. A file or `TAGS` is required — running `make test` without arguments will error.

```bash
# Run a specific file
make test .features-gen/features/oa/oa.feature.spec.js

# Run from a specific line
make test .features-gen/features/abacus/abacusContract2.feature.spec.js:51

# Run by tag
make test TAGS="@qa_smoke"

# Run by multiple tags
make test TAGS="@qa_smoke or @qa_regression"

# Run with Playwright Inspector (debug mode)
make debug .features-gen/features/oa/oa.feature.spec.js
make debug TAGS="@id_ee732b50acbed8c72ede09e0530355fa9dd4f0dadbb95962149847f742c1fa71"

# List all targets
make help
```

## Running Tests in Docker

The repo includes a `Dockerfile` and `docker-compose.yml` for running tests using the official Playwright image.

### 1. Build the image

```bash
docker compose build playwright-tests
```

### 2. To run all tests

```bash
docker compose run --rm playwright-tests "pnpm playwright test"
```

### 3. To run a specific test file

```bash
docker compose run --rm playwright-tests "pnpm playwright test .features-gen/features/workstation/login.feature.spec.js --reporter=html"
```

### 4. Filter by tags

```bash
docker compose run --rm -e TAGS="@qa_smoke" playwright-tests "pnpm playwright test --reporter=html"
```

### 5. View results

After the run, open the HTML report locally:

```
open playwright-report/index.html
```

---

### Watch & Auto-Regenerate (`pnpm dev`)

Quick watch mode for feature authoring: regenerates specs on any change in `features/`.

```bash
pnpm dev
```

Keep it running while you edit `.feature` files; run Playwright tests in another terminal.

### Automatic Scenario ID Tagging

A pre-commit hook automatically adds unique `@id_` tags to scenarios missing them when you commit `.feature` files.

### Auto-fix formatting and linting

```bash
# Run on staged files
pnpm lint-staged
```

## IDE setup

For a better development experience, we recommend using the following VSCode extensions:

- [Playwright Test for VSCode](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright)
- [Cucumber (Gherkin) Full Support](https://marketplace.visualstudio.com/items?itemName=alexkrechik.cucumberautocomplete)

To configure Cucumber support in VSCode, add the following to your `.vscode/settings.json`:

```json
{
  "cucumberautocomplete.steps": ["./src/bdd-steps/**/*.ts"],
  "cucumberautocomplete.syncfeatures": "./features/**/*.feature"
}
```

### JetBrains IDEs (WebStorm, IntelliJ IDEA, etc.)

For JetBrains IDEs, you can use the built-in support for Playwright with the cucumber.js and Gherkin plugins. (see )
https://www.notion.so/Setting-Up-a-Cucumber-IDE-Plugin-1a197177520f801e8ae3cfd70c28d61c for plugin installation and configuration.)

You can use the built-in playwright runner for Jetbrains IDEs. Copy the .env.shadow file to .env and set the environment variables,
then configure a new Playwright run configuration:

1. Open the Run/Debug Configurations dialog.
2. Click the "+" button and select "Playwright Test".
3. Set the "Node interpreter" to your Node.js installation.
4. Set the "Working directory" to the root of your project.
5. Set the "Test Directory" to `playwright-tests/.features-gen`

Then use the TAGS environment variable to run specific tests.

## Project Structure & Organization

```
playwright-tests/
├── features/                # BDD Gherkin feature files
├── framework/               # Framework utilities and scripts
│   └── scripts/             # Automation scripts (AWS secrets, etc.)
├── src/
│   ├── apps/                # Modular app factories (one folder per app)
│   │   └── queries.ts/      # Per-app queries.ts files
│   │   └── requests.ts/     # Per-app requests.ts files
│   ├── bdd-steps/           # BDD step definitions (general & app-specific)
│   │   └── hooks.ts/        # Per-app hooks.ts files
│   ├── components/          # Reusable UI components
│   ├── fixtures/            # Test fixtures
│   ├── helpers/             # Utility functions
│   ├── pages/               # Page Object Model classes (per app)
│   ├── types/               # App-specific and shared types
│   └── utils/               # Common utilities (config, db, AWS, etc.)
│   │   └── apis/            # apis used for different apps, each one has its own type file
├── playwright.config.ts     # Playwright configuration
├── Dockerfile               # Docker configuration
├── Makefile                 # Build automation
└── package.json             # Project dependencies and scripts
```

### Key Concepts

- **Config Loading:**
  - All config is loaded via `SecretsManagerClient.getConfig()` (see `src/utils/aws/SecretsManagerClient.ts`).
- **Database Client:**

  - Use `DatabaseClient` and `createDatabaseConfigs` from `src/utils/database.ts` for DB access.
  - DB config is loaded from secrets/config and passed to the client.

- **Apps, Queries, and Types:**

  - Each app in `src/apps/` has its own folder, `index.ts` (factory),`requests.ts` for app-specific request interception and `queries.ts` for app-specific queries.
  - Types for each app live in `src/types/{appName}.ts`.
  - APIs are modularized: each API file has a corresponding type file if needed.

- **Hooks:**
  - General hooks are in `src/bdd-steps/hooks.ts`.
  - App-specific hooks can be placed in `src/bdd-steps/{app}/hooks.ts`.

---

## Adding a New App (Suite Application)

1. **Create the App Factory:**

   - Go to `src/apps/` and create a folder for your app (e.g. `settings`).
   - Add `index.ts` (factory) and `requests.ts` (for request interception/helpers).

   Example `index.ts`:

   ```ts
   import { Page } from "@playwright/test";
   import { requests } from "./requests";

   export function createSettingsApp(page: Page, opts: { host: string }) {
     return {
       page,
       requests,
     };
   }
   ```

   Example `requests.ts`:

   ```ts
   export const requests = {};
   ```

2. **Register the App Factory:**

   - Edit `src/apps/index.ts` and add your factory to `appFactories`.

   ```ts
   import { createSettingsApp } from "@/src/apps/settings";

   export const appFactories = {
     auth: (page: Page, opts) => ({ page, loginPage: new LoginPage(page, opts), requests: {} }),
     settings: createSettingsApp,
   } satisfies Record<
     string,
     (
       page: Page,
       opts: { host: string },
     ) => { page: Page; requests: { [key: string]: string | RegExp }; [key: string]: unknown }
   >;

   export type Apps = {
     [K in keyof typeof appFactories]: ReturnType<(typeof appFactories)[K]>;
   };
   ```

3. **Add Pages:**

   - Create a folder in `src/pages/{app}` and add your page classes (e.g. `HomePage.ts`).
   - Example:

   ```ts
   import { BasePage, type BaseUrlOptions } from "@/src/BasePage";
   import { type Page } from "@/src/playwright-bdd";

   export class SettingsHomePage extends BasePage<BaseUrlOptions> {
     constructor(page: Page, defaultOptions: BaseUrlOptions = {} as BaseUrlOptions) {
       super(page, { ...defaultOptions, path: "/" });
     }
   }
   ```

   - Add the page to your app factory:

   ```ts
   import { SettingsHomePage } from "@/src/pages/settings/HomePage";
   // ...
   export function createSettingsApp(page: Page, opts: { host: string }) {
     return {
       page,
       requests,
       homePage: new SettingsHomePage(page, opts),
     };
   }
   ```

4. **Add Steps:**

   - Create a folder in `src/bdd-steps/{app}` and add your step files (e.g. `settings.ts`).
   - Example:

   ```ts
   import { expect, Then } from "../../playwright-bdd";

   Then("I see that the settings home page works", async ({ apps }) => {
     apps.settings.homePage.verifyPageWorks();
   });
   ```

5. **Add Base URL Override Environment Variable:**

   - Edit `env.ts` and add a new base URL override for your app in the zod schema:

   ```ts
   const envSchema = z.object({
     // ... existing fields
     BASE_URL_OVERRIDE_YOURAPP: baseURLOverride,
   });
   ```

   This allows running tests against custom environments or non-standard local ports:

   ```bash
   # Test against custom environment
   BASE_URL_OVERRIDE_YOURAPP=https://your-app.staging.com pnpm playwright test

   # Test against local dev server on custom port
   BASE_URL_OVERRIDE_YOURAPP=http://localhost:3001 pnpm playwright test
   ```

   Alternatively, add the environment variable to your `.env` file:

   ```bash
   # .env
   BASE_URL_OVERRIDE_YOURAPP=http://localhost:3001
   ```

6. **Provision AWS Secrets (via Terraform):**

   Two secrets are required for a new app and must be created in Terraform before tests will run:

   - **Auth0 credentials** — required for silent login (used in QA to bypass the browser login flow):

     Add the new app to [`qa/auth0-secrets/variables.tf`](https://github.com/theorchard/terraform-infra/blob/master/qa/auth0-secrets/variables.tf).

     Path: `${env}/auth0-secrets/{appKey}` (e.g. `qa/auth0-secrets/settings`)

     Shape:

     ```json
     {
       "client_id": "...",
       "client_secret": "..."
     }
     ```

     Without this secret, any test that calls `Auth0Client.create("{appKey}")` will fail at startup.

   - **User data** — required for tests that look up test users by label:

     Add the new app to [`qa/e2e-test-secrets/variables.tf`](https://github.com/theorchard/terraform-infra/blob/master/qa/e2e-test-secrets/variables.tf).

     Path: `${env}/e2e-test-secrets/user-data/{appKey}-user-data` (e.g. `qa/e2e-test-secrets/user-data/settings-user-data`)

     Shape is app-specific but typically contains a map of user labels to credentials used in step definitions.

7. **Add Queries and Types:**
   - Add queries to `src/apps/{app}/queries/queries.ts`.
   - Add types to `src/types/{app}.ts`.

---

## Hooks

- General hooks: `src/bdd-steps/hooks.ts`
- App-specific hooks: `src/bdd-steps/{app}/hooks.ts`

---

## API & Type Modularization

- Each API file in `src/utils/apis/` should have a corresponding type file in `src/types/` if needed.
- App-specific types should live in `src/types/{app}.ts`.
- Queries for each app go in `src/apps/{app}/queries.ts`.

---

## Database Client Usage

- Use `DatabaseClient` and `createDatabaseConfigs` from `src/utils/database.ts`.
- DB config is loaded from secrets/config and passed to the client.
- See code comments in `src/utils/database.ts` for usage examples.

---

## Onboarding & Best Practices

- Always use the provided config/context/database setup; do not hardcode secrets or config.
- Keep APIs, queries, and types modular and per-app where possible.
- Add new hooks, queries, and types in the appropriate per-app folders.
- See code comments for further details and examples.

## AWS Secrets Management

The project includes a CLI tool for managing AWS Secrets Manager secrets used by the test suite.

### Prerequisites

You need to have a valid AWS session to access Secrets Manager. Use AWSUME before running the CLI or tests.

### Update Secret CLI

The `update-secret` command allows you to download, edit, and upload secrets stored in AWS Secrets Manager.

#### Available Commands

**Interactive Mode (Default)**

```bash
pnpm update-secret
# or
pnpm update-secret interactive
```

Presents an interactive menu to select which secret to update.

**Update Playwright Configuration**

```bash
pnpm update-secret config
```

Directly downloads and updates the `playwright.env.json` configuration file.

**Update User Data Secrets**

```bash
pnpm update-secret user-data
```

Interactive mode for updating application-specific user data secrets.

You can also specify options to skip prompts:

```bash
pnpm update-secret user-data -a collaborators -e qa
pnpm update-secret user-data -a workstation -e prod
```

Options:

- `-a, --application <app>` - Application name (e.g., collaborators, workstation, insights)
- `-e, --environment <env>` - Environment: `qa` or `prod`

#### Workflow

1. The CLI downloads the secret from AWS Secrets Manager to a temporary file (`awsSecretToUpload.json`)
2. You edit the file in your IDE
3. The CLI validates your changes are valid JSON
4. The CLI uploads the updated secret back to AWS Secrets Manager
5. The temporary file is cleaned up

#### Help

```bash
pnpm update-secret --help
pnpm update-secret user-data --help
```

---

## CLI Debug Mode

[`playwright-cli`](https://github.com/microsoft/playwright-cli) is a command-line tool that lets an AI coding agent interact with a running browser. This framework includes a debug mode that keeps the browser alive after a test completes so that a Claude Code session can attach to it, inspect the page, and help build or debug step definitions against the real app state.

### Prerequisites

Install the `playwright-cli` skills into your Claude Code session:

```bash
pnpm playwright-cli install --skills
```

### Enabling CLI debug mode

> **Note:** Only a single test is supported at a time — use an `@id_` tag to target a specific scenario.

Set `PLAYWRIGHT_CLI_DEBUG_SESSION` to the session name you want to bind to:

```bash
PLAYWRIGHT_CLI_DEBUG_SESSION=my-session TAGS=@id_<scenario-id> pnpm playwright test
```

The test will run, then pause after completion and print a signal command:

```
🔗 Browser bound to "my-session".
   To continue: touch ./tmp/playwright-debug-continue
```

### Connecting Claude to the browser

With the test paused, ask Claude to attach to the session by name:

```
Connect to my-session and take a snapshot
```

Claude will use `playwright-cli` to attach to the running browser and can take snapshots, interact with elements, read console errors, and inspect network requests — all against the live page state at the point the test stopped.

When you're done, run the `touch` command printed by the test to release the browser and complete teardown.