# terraform-infra (staging copy)

> **This is an *anticipated* scaffold, not the live infra.** It mirrors the
> `permissions-platform/{prod,qa}` layout from the real `theorchard/terraform-infra`
> monorepo so the Bedrock access for `skill-eval-runner` can be reviewed alongside the
> library. When ready, move `permissions-platform/<env>/skill-eval-runner/` into the
> actual `terraform-infra` repo and open a PR there.

## What this provisions

A dedicated, **inference-only** Bedrock identity in each Permissions Platform account so
the `skill-eval-runner` harness can run `--provider bedrock` evals:

- `aws_iam_policy` `<env>-skill-eval-runner-bedrock-policy` — allows
  `bedrock:InvokeModel` + `bedrock:InvokeModelWithResponseStream` on the foundation-model
  **and** cross-Region `inference-profile` ARNs for each Claude model the runner uses
  (opus-4-6, sonnet-4-6, haiku-4-5). Mirrors `qa/ows-coda/iam.tf`.
- An attachment of that policy to the shared **`generic-engineer-role`** (looked up by
  `data` — it already exists and supports cross-account access), so engineers can run
  `skill-eval --provider bedrock` without a bespoke role. Mirrors
  `qa/pdp-backfill/bucket/generic-engineer-role.tf`.

Account IDs are never hardcoded — the inference-profile ARNs are built from
`data.aws_caller_identity.current.account_id`, so the same code applies in either account.

```
permissions-platform/
  prod/skill-eval-runner/{versions,main,variables,iam,outputs}.tf   # environment=prod, prod-…-terraform-state
  qa/skill-eval-runner/{versions,main,variables,iam,outputs}.tf     # environment=qa,   qa-…-terraform-state
```

Each directory is a self-contained Terraform root (the repo convention), with its own S3
backend and `provider "aws"` carrying `terraform-default-tags`.

## Before you apply — review these

1. **`generic-engineer-role`** — the policy attaches to this shared, pre-existing cross-account
   role (looked up via `data`, not created here). Confirmed present in **both** PP accounts
   (qa `591204808501`, prod `031099521156`). To also let CI invoke Bedrock, attach the exported
   `bedrock_policy_arn` to the CI/Jenkins role.
2. **`bedrock_model_id_patterns`** — keep in lockstep with the model presets in
   `skill_eval_runner/providers/bedrock.py`. The exact version-dated model IDs must be
   **enabled** in the account (Bedrock console → Model access) — Terraform grants invoke
   permission but does not enable model access.
3. **Region** — constrained to `us-east-1` / `us-west-2` (Bedrock availability), matching the
   runner's `BEDROCK_REGION` default.

## Usage (from the real terraform-infra repo, on VPN)

```sh
cd permissions-platform/qa/skill-eval-runner
terraform init
terraform plan
terraform apply
```

Then point the runner at the account: set `BEDROCK_REGION` and use AWS creds for the
`generic-engineer-role` (or any role with `bedrock_policy_arn` attached), and run
`skill-eval validate --provider bedrock`.

### Local AWS profiles

The `generic-engineer-role` is assumed via these profiles (per-engineer `~/.aws/config`):

| Account | AWS profile                          | Account ID     |
|---------|--------------------------------------|----------------|
| qa      | `permissions-platform-qa-generic`    | `591204808501` |
| prod    | `permissions-platform-prod-generic`  | `031099521156` |

```sh
# Run a qa Bedrock eval as the generic engineer:
AWS_PROFILE=permissions-platform-qa-generic BEDROCK_REGION=us-east-1 \
  skill-eval validate --provider bedrock
```