---
name: python-patch-vulnerabilities
description: This skill should be used when the user wants to patch Dependabot security vulnerabilities, fix open security alerts in GitHub repos, or run the vulnerability patching workflow across a list of repositories.
version: 1.0.0
user-invocable: true
---

# Patch Vulnerabilities

Patches open Dependabot security vulnerabilities across a list of GitHub repositories. Creates one branch + commit + PR per affected dependency file, then waits for CI before moving on.

## Input

Read `repos.txt` from the **current working directory**. Each line is either a full GitHub URL (`https://github.com/owner/repo`) or `owner/repo`. Skip blank lines and lines starting with `#`.

## Setup (once at start)

```bash
GH_USER=$(gh api user --jq '.login')
REPOS_DIR="${REPOS_DIR:-$HOME/repos}"
mkdir -p "$REPOS_DIR"
AWSUME_CMD=$(pyenv which awsume 2>/dev/null || command -v awsume 2>/dev/null || echo "")
if [[ -z "$AWSUME_CMD" ]]; then
  echo "[WARNING] awsume not found — Docker pulls from ECR may fail. Install awsume or set AWSUME_CMD manually."
fi
```

## Phase 1 — Repo setup (per repo)

1. Parse `owner/repo` from the line (strip URL prefix if present).
2. Local path: `$REPOS_DIR/{repo-name}` where `repo-name` is the last path segment.
3. If the directory does **not** exist:
   ```bash
   cd "$REPOS_DIR"
   gh repo fork {owner}/{repo} --clone --remote-name upstream
   # gh repo fork clones the fork as origin and adds upstream automatically.
   # If the cloned directory name doesn't match {repo-name}, rename it.
   ```
4. Verify remotes:
   ```bash
   git -C $REPOS_DIR/{repo-name} remote get-url upstream   # should be the theorchard/... URL
   git -C $REPOS_DIR/{repo-name} remote get-url origin     # should be the user's fork
   ```
5. Fetch upstream and detect the default branch:
   ```bash
   git -C $REPOS_DIR/{repo-name} fetch upstream
   DEFAULT_BRANCH=$(gh api repos/{owner}/{repo} --jq '.default_branch')
   ```

## Phase 2 — Alert discovery (per repo)

1. Fetch open alerts with available patches:
   ```bash
   gh api 'repos/{owner}/{repo}/dependabot/alerts?state=open&per_page=100' \
     -H "Accept: application/vnd.github+json" \
     --jq '[.[] | select(.security_advisory.vulnerabilities[].first_patched_version != null)]'
   ```
   **Note:** Use `?state=open` as a URL query parameter — do **not** use `-f state=open` (that sends it as a request body and causes a 404).
2. Group the filtered alerts by `dependency.manifest_path`.
3. If no patchable alerts remain, log `[{repo}] No patchable alerts — skipping` and continue to next repo.

## Phase 3 — One PR per manifest file

For each unique `manifest_path` (e.g., `requirements.txt`, `lambda/my-function/requirements.txt`, `poetry.lock`):

### a. Create branch

Slugify the full manifest path (replace `/` and `.` with `-`):
```bash
SLUG=$(echo "{manifest_path}" | tr '/.' '-')
BRANCH="MAINT-vulnerabilities-${SLUG}-$(date +%Y%m%d)"
# If branch already exists, append -2, -3, etc.
git -C $REPOS_DIR/{repo-name} checkout -b "$BRANCH" upstream/$DEFAULT_BRANCH
```

### b. Update dependencies

**Determine the manifest directory** (the directory containing the manifest file). For root-level files this is the repo root; for `lambda/my-function/requirements.txt` it is `$REPOS_DIR/{repo-name}/lambda/my-function`.

**requirements.txt** (manifest filename is exactly `requirements.txt` — last path segment matches, e.g. `requirements.txt` or `lambda/func/requirements.txt`):

> **Note:** `update-requirements.sh` requires the file to contain `# top level dependencies` and `# sub dependencies` section headers. Files named anything other than `requirements.txt` (e.g. `requirements-dev.txt`) are flat pinned lists — see **Flat pinned requirements files** below.

- Check for `Dockerfile` in the **manifest directory**:
  ```bash
  ls {manifest_dir}/Dockerfile
  ```
- If found: copy the bundled `update-requirements.sh` script to a temp path and run it from the manifest directory:
  ```bash
  cp "${CLAUDE_SKILL_DIR}/scripts/update-requirements.sh" /tmp/update-requirements.sh
  chmod +x /tmp/update-requirements.sh
  cd {manifest_dir} && /tmp/update-requirements.sh
  ```
- If no Dockerfile: log `[{repo}] WARNING: No Dockerfile found in {manifest_dir} — cannot regenerate {manifest_path}. Skipping this file.` and continue to next manifest.

**Flat pinned requirements files** (any `.txt` manifest whose filename is NOT exactly `requirements.txt`, e.g. `requirements-dev.txt`, `requirements-test.txt`):

These are flat lists of pinned versions without the structured sections that `update-requirements.sh` expects. Update them manually:

1. Read the manifest file to see all pinned packages.
2. **Update the alerted package(s)** to at least `first_patched_version`. Keep the pinned-version style (`pkg==X.Y.Z`) consistent with the file.
3. **Check for companion packages** that must also be updated for compatibility. This is especially important for test frameworks — a major version bump (e.g. `pytest 7→9`) often requires updating closely related packages (`pytest-cov`, `pytest-mock`, `flexmock`, etc.) to versions that support the new major. Check each companion's changelog or PyPI page for the minimum version compatible with the new major.
4. **Run the test service** to validate:
   ```bash
   COMPOSE_FILE=$(ls {manifest_dir}/docker-compose.yml {manifest_dir}/docker-compose.yaml 2>/dev/null | head -1)
   LINT_SERVICE=$(grep -E "^\s+[a-zA-Z_-]*(unit.lint|lint.and.test)[a-zA-Z_-]*\s*:" "$COMPOSE_FILE" 2>/dev/null | head -1 | tr -d ' :')
   source $AWSUME_CMD prod && cd {manifest_dir} && docker compose run --rm --build ${LINT_SERVICE}
   ```
5. **Iterate on test failures** — major version bumps often break tests due to API changes. Two common patterns:
   - *Import style*: e.g. `flexmock 0.12.x` removed the module-level `__call__` shortcut — `import flexmock; flexmock(obj)` must become `from flexmock import flexmock; flexmock(obj)`. Identify which test files use the old pattern with grep, then fix them all (Python one-liner or `sed` loop).
   - *Removed/renamed API*: check the package's migration guide or `CHANGELOG`.

   Fix consistently across all affected files, re-run until all tests pass, then proceed.
6. **Stage test files** alongside the manifest in step d — the dependency bump and the test fixes are one logical change.

**poetry.lock / pyproject.toml** (manifest_path ends with `poetry.lock` or `pyproject.toml`):
- The manifest directory is where the docker-compose file and `poetry.lock` live.
- Find the docker-compose file (check both `.yml` and `.yaml`):
  ```bash
  COMPOSE_FILE=$(ls {manifest_dir}/docker-compose.yml {manifest_dir}/docker-compose.yaml 2>/dev/null | head -1)
  ```
- Scan for a lockfile-update service by looking for common keywords (`update-lockfile`, `update_lockfile`, `update-poetry`, `lockfile`):
  ```bash
  LOCKFILE_SERVICE=$(grep -E "^\s+[a-zA-Z_-]*(update.lock|lockfile)[a-zA-Z_-]*\s*:" "$COMPOSE_FILE" 2>/dev/null | head -1 | tr -d ' :')
  ```
- If found, run it **once per affected package** (pass the package name as a positional argument — omitting it would upgrade all packages, not just the targeted ones):
  ```bash
  for pkg in {affected_package_names}; do
    source $AWSUME_CMD prod && cd {manifest_dir} && docker compose run --rm --build {LOCKFILE_SERVICE} "$pkg"
    # Check if the package version actually changed
    CURRENT_PKG_VER=$(grep -A2 "name = \"$pkg\"" {manifest_dir}/poetry.lock | grep "version =" | head -1 | grep -oE '"[^"]+"' | tr -d '"')
    PATCHED_VER="{first_patched_version for $pkg from the Dependabot alert}"
    if [ "$CURRENT_PKG_VER" version-lt "$PATCHED_VER" ]; then
      # Version didn't advance — pyproject.toml constraint likely blocking it
      CONSTRAINT_LINE=$(grep -in "\"$pkg" {manifest_dir}/pyproject.toml | head -1)
      if [ -n "$CONSTRAINT_LINE" ]; then
        log "[{repo}] {pkg}: locked at $CURRENT_PKG_VER, needs $PATCHED_VER — constraint in pyproject.toml is blocking. Relaxing upper bound..."
        # Determine the new minimum upper bound: bump the major version of first_patched_version by 1
        # e.g. patched=9.0.3 → new upper bound <10.0
        # Edit pyproject.toml: find the line containing the package name and remove or raise the upper bound
        # Use sed or Python to update the line in-place, e.g.:
        #   python3 -c "
        #   import re, sys
        #   content = open('{manifest_dir}/pyproject.toml').read()
        #   # Match lines like:  \"pytest (>=8.3,<9.0)\"  or  pytest = \">=8.3,<9.0\"
        #   # Remove any upper-bound clause (,<X.Y or ,<=X.Y) for this package
        #   pattern = r'(?i)(\"$pkg[^\"]*\([^)]*),<[0-9]+[^)]*(\))'
        #   updated = re.sub(pattern, r'\1\2', content)
        #   open('{manifest_dir}/pyproject.toml', 'w').write(updated)
        #   "
        # Re-run the lockfile update after relaxing the constraint
        source $AWSUME_CMD prod && cd {manifest_dir} && docker compose run --rm --build {LOCKFILE_SERVICE} "$pkg"
      else
        log "[{repo}] WARNING: {pkg} at $CURRENT_PKG_VER, first_patched=$PATCHED_VER — no direct constraint found in pyproject.toml (likely transitive). Cannot force update. Skipping this package."
      fi
    fi
  done
  ```
  Where `{affected_package_names}` is the list of package names from the Dependabot alerts for this manifest.
- If not found but Dockerfile has an `update-lockfile` or `poetry_update` target, build and run directly:
  ```bash
  cd {manifest_dir}
  docker build --platform linux/amd64 --target update-lockfile -t tmp-ulk-{slug} . -q
  docker run --rm -v "$PWD/poetry.lock:/var/app/poetry.lock" tmp-ulk-{slug}
  ```
  (adjust the volume mount path to match the `WORKDIR` in the Dockerfile — usually `/var/app/poetry.lock` or `/var/task/poetry.lock`)
- If neither Docker option is available, log a warning and skip. **Do not use local `poetry` commands.**
- After the update runs, also check whether `pyproject.toml` was modified (it sometimes gets updated alongside `poetry.lock`). Stage it in step c if so.

### c. Verify the lockfile update succeeded

Always check `git status` after the update step to confirm the file was actually modified. If there are no changes, the dependencies were already up to date — skip committing and note it in the summary.

```bash
git -C $REPOS_DIR/{repo-name} status --short
```

**Verify patched versions** — for each alert that was targeted, confirm the updated file actually contains a version at or above the `first_patched_version` from the alert. For `poetry.lock`, grep for the package name and inspect the version line. For `requirements.txt`, check the pinned or minimum version. If any targeted package is still below the patched version, log a warning and do **not** open a PR for this manifest.

```bash
# Example check for a package named {pkg} with first_patched_version {patched_ver}:
grep -A2 "name = \"{pkg}\"" {manifest_dir}/poetry.lock | grep "version ="
# Confirm the version is >= {patched_ver} before proceeding.
```

**For `poetry.lock` manifests only** — run the repo's lint/test service via Docker to catch lockfile consistency errors **before committing or pushing**:
```bash
# Find the lint/test service name (usually unit-lint or lint-and-test)
LINT_SERVICE=$(grep -E "^\s+[a-zA-Z_-]*(unit.lint|lint.and.test)[a-zA-Z_-]*\s*:" "$COMPOSE_FILE" 2>/dev/null | head -1 | tr -d ' :')
source $AWSUME_CMD prod && cd {manifest_dir} && docker compose run --rm --build ${LINT_SERVICE}
```
If no lint service is found, skip this step.

If the lint/test run fails:
- *"pyproject.toml changed significantly since poetry.lock was last generated"* → the `poetry.lock` needs full regeneration. **Immediately re-run the lockfile update step** (without a package argument, so `poetry lock` regenerates the full lockfile), then run the lint/test service again to confirm it passes before proceeding.
- Failures unrelated to the dependency update → note them and proceed with the PR anyway.

**For `requirements.txt` manifests** — skip this step; `update-requirements.sh` already validates the update by running pip install inside Docker.

**For flat pinned requirements files** — tests must pass before committing (covered in step b above). Do not open a PR until the test service exits 0.

### d. Commit

Stage the manifest file, any associated lockfile, and `pyproject.toml` if it was modified:
```bash
git -C $REPOS_DIR/{repo-name} add {manifest_path}
# Also stage pyproject.toml and/or poetry.lock if changed alongside the manifest:
git -C $REPOS_DIR/{repo-name} status --short | grep -E "pyproject.toml|poetry.lock" | awk '{print $2}' | xargs -r git -C $REPOS_DIR/{repo-name} add
# For flat pinned requirements files: also stage any test files fixed for dependency API compatibility:
git -C $REPOS_DIR/{repo-name} status --short | grep "tests/" | awk '{print $2}' | xargs -r git -C $REPOS_DIR/{repo-name} add
git -C $REPOS_DIR/{repo-name} commit -m "$(cat <<'EOF'
MAINT: Address vulnerabilities in {manifest_path}

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EOF
)"
```

### e. Push and open PR

Fetch alert details for the PR body — filter by manifest path using shell-level grep rather than `jq --arg` (which doesn't work with `gh --jq` in zsh):
```bash
ALL_ALERTS=$(gh api 'repos/{owner}/{repo}/dependabot/alerts?state=open&per_page=100' \
  -H "Accept: application/vnd.github+json" \
  --jq '.[] | select(.security_advisory.vulnerabilities[].first_patched_version != null) | "- **\(.dependency.package.name)** (\(.dependency.package.ecosystem)): \(.security_advisory.summary) — \(.security_vulnerability.severity) — \(.security_advisory.cve_id // .security_advisory.ghsa_id) [\(.dependency.manifest_path)]"' \
  | tr -d '"')
MANIFEST_ALERTS=$(echo "$ALL_ALERTS" | grep "\[{manifest_path}\]" | sed 's/ \[.*\]$//' | sort -u)
```

Then create the PR:
```bash
git -C $REPOS_DIR/{repo-name} push -u origin "$BRANCH"

PR_URL=$(gh pr create \
  --title "MAINT: Address vulnerabilities in {manifest_path}" \
  --base $DEFAULT_BRANCH \
  --repo {owner}/{repo} \
  --head "${GH_USER}:${BRANCH}" \
  --body "Addresses the following Dependabot alerts:

${MANIFEST_ALERTS}

🤖 Generated by [Claude Code](https://claude.ai/code)")
echo "Opened PR: $PR_URL"
```

### f. Wait for CI

```bash
# Retry --watch until checks register (it exits immediately with "no checks reported" if CI hasn't started yet)
for i in 1 2 3 4 5; do
  sleep 15
  gh pr checks "$PR_URL" --watch 2>&1 && break || true
done
```

After the watch loop exits, check for failures:
```bash
FAILED=$(gh pr checks "$PR_URL" --json name,state --jq '[.[] | select(.state == "FAILURE")] | length' 2>/dev/null || echo "0")
```
- If `FAILED > 0`: print the failing check names, log `[{repo}] CI failed on {manifest_path} — stopping this repo`, and move on to the next repo (skip remaining manifest files for this repo).
- If all passed: log `[{repo}] PR merged/passed for {manifest_path}` and continue to next manifest.
- If no checks ever register after the retry loop: log `[{repo}] No CI checks found for {manifest_path} — treating as passed` and continue.

## Phase 4 — Summary

After processing all repos, print a summary table:

```
Repo                        | Manifest                        | PR URL          | CI
----------------------------|----------------------------------|-----------------|----
theorchard/ows-assets       | requirements.txt                | https://...     | PASS
theorchard/ows-assets       | lambda/ingest/requirements.txt  | https://...     | PASS
theorchard/lambda-auth      | poetry.lock                     | (skipped)       | n/a
```

## Warnings and edge cases

- **Patchable filter**: Only process alerts where at least one entry in `security_advisory.vulnerabilities` has a non-null `first_patched_version`. Alerts with no patch available are logged but skipped.
- **Branch collision**: If `MAINT-vulnerabilities-{slug}-{date}` already exists locally or on origin, append `-2`, `-3`, etc.
- **Missing tooling**: Always warn clearly (repo + manifest path) rather than failing the entire run. One bad manifest should not block other repos.
- **Working directory**: Always use `git -C $REPOS_DIR/{repo-name}` or `cd {dir}` explicitly — never assume cwd.
- **Docker only**: Use Docker/docker-compose and `update-requirements.sh` to update dependencies — do not run `poetry update` locally. Always use `--build` with `docker compose run` (`docker compose run --rm --build {service}`) to ensure the image is up-to-date — without it, stale cached images can cause the container to run without writing changes. Fall back to `docker build --target ... && docker run` only when no compose service exists.
- **AWS credentials**: Prepend all Docker commands with `source $AWSUME_CMD prod &&` so Docker can pull ECR images (`$AWSUME_CMD` is resolved at setup from `pyenv which awsume` or `command -v awsume`). Do not run `aws ecr get-login-password | docker login` — awsume sets the env vars automatically. If awsume requires MFA, tell the user to run `! awsume prod` in the prompt first.
- **Stale lockfile**: If a `poetry.lock` update fails with *"pyproject.toml changed significantly since poetry.lock was last generated"*, the upstream `pyproject.toml` was modified after the lockfile was last generated. **Automatically re-run the lockfile update step without a package argument** so `poetry lock` fully regenerates the lockfile. Then re-run the lint/test service to confirm it passes before committing. Always stage `pyproject.toml` alongside `poetry.lock` if it was modified.
- **Docker parallelism**: Running more than 3–4 Docker containers simultaneously can overwhelm Colima's VM and cause containers to be killed mid-run (output will end abruptly at "Resolving dependencies..." with no lockfile written). Always verify `git status` after each update. Process repos in small batches — or sequentially — rather than all at once.
- **Flat requirements files**: Files like `requirements-dev.txt` are flat pinned lists — `update-requirements.sh` will fail with "Missing '# sub dependencies' section" if run against them. Use the **Flat pinned requirements files** path instead: update versions manually, bump compatible companion packages, and validate with the test service before committing.
- **Test code fixes after major version bumps**: When a test-only dependency (pytest, flexmock, etc.) has a major version bump, the new version may remove or rename APIs used in test files. Always run the test service after updating test dependencies. If tests fail, read the traceback, identify the pattern across affected files (e.g. grep for the broken import), fix consistently, and include the test fixes in the same commit as the requirements change. A PR that updates test deps but leaves tests broken is not acceptable.