---
name: analyze-cost-anomaly
description: Investigate an AWS cost anomaly and produce a root-cause report. Use when the user wants to analyze an AWS cost spike/anomaly, asks why AWS costs increased, or wants to review recent Cost Anomaly Detection alerts.
argument-hint: "[anomaly-id | YYYY-MM-DD..YYYY-MM-DD | empty to list anomalies]"
---

# AWS Cost Anomaly Root-Cause Analysis

You are acting as a FinOps analyst. Investigate the anomaly with the helper
scripts below, then write a root-cause report. All scripts print compact JSON
and accept `--profile NAME` if the user works with a non-default AWS profile.

The scripts are bundled with this plugin. Invoke them via the
`${CLAUDE_PLUGIN_ROOT}` environment variable so they resolve no matter the
user's working directory, e.g.:
`uv run ${CLAUDE_PLUGIN_ROOT}/skills/analyze-cost-anomaly/scripts/list_anomalies.py`.
They are [PEP 723](https://peps.python.org/pep-0723/) scripts (inline
dependencies); `uv run` resolves and caches `boto3` on first use — no virtualenv
to set up. They need AWS credentials via the standard chain and read-only
permissions: `ce:GetAnomalies`, `ce:GetCostAndUsage`,
`ce:GetSavingsPlansUtilization`, `ce:GetSavingsPlansCoverage`,
`cloudtrail:LookupEvents`, and the `describe-reserved-*` /
`describe-savings-plans` calls.

Cost note: each Cost Explorer call costs $0.01. Be deliberate — a typical
investigation needs 3–8 calls, not dozens. CloudTrail and reservation describe
calls (step 4, step 5 without `--savings-plans`) are free; the Savings Plans
option in step 5 adds ~$0.02.

## Arguments

- **No argument** → run step 1, show the anomalies as a short table (id, dates,
  impact, AWS hint), and ask the user which one to investigate.
- **Anomaly id** (long alphanumeric token) → find it via step 1, then investigate.
- **Date range `YYYY-MM-DD..YYYY-MM-DD`** → skip step 1; treat the range as the
  analysis period and investigate the spend change directly (this works even
  when AWS detected no anomaly).

## Procedure

In the commands below, `$S` stands for
`${CLAUDE_PLUGIN_ROOT}/skills/analyze-cost-anomaly/scripts`.

1. **Fetch anomalies**
   `uv run $S/list_anomalies.py [--days 60]`
   Note the anomaly window, total impact, and `root_cause_hints` — AWS often
   already names the service/region/usage type. Treat hints as a starting
   point to verify, not as the conclusion.

2. **Baseline comparison by service**
   `uv run $S/cost_breakdown.py --start <anomaly-start> --end <anomaly-end>`
   The output compares the period against the prior 14 days. Identify which
   service(s) account for the delta, and use `daily_totals_usd` to pinpoint the
   exact day the spike began.

3. **Drill down on the spiking service(s)**
   Re-run with `--group-by USAGE_TYPE --filter-service "<exact service name from step 2>"`,
   and `--group-by REGION` if region matters. Usage type is usually the most
   diagnostic dimension (e.g. `NatGateway-Bytes`, `DataTransfer-Out-Bytes`,
   `BoxUsage:p4d.24xlarge`). If the account uses cost-allocation tags,
   `--group-by TAG:<key>` can attribute the spend to a team/app.

4. **Find what changed**
   `uv run $S/cloudtrail_events.py --start <day-before-spike> --end <spike-day+1>`
   Look for write events that plausibly explain the usage-type increase
   (RunInstances, CreateNatGateway, CreateCluster, PutLifecycleConfiguration,
   UpdateFunctionConfiguration, ...). Use `--event-name X` to zoom in, and
   `--region` to match the region found in step 3. Note the `username` —
   "who did it" is part of the answer.

5. **Check reservation / Savings Plan coverage** — do this *whenever the spike
   is in a reservable service* (RDS, EC2/Compute, ElastiCache, OpenSearch,
   Redshift), and *especially* when step 4 found no CloudTrail change.
   `uv run $S/reservation_coverage.py --near <spike-day> --region <region>`
   A **coverage lapse** — a Reserved Instance or Savings Plan expiring the day
   before the step — is a top cause of step-change cost increases and leaves
   **no CloudTrail event**, because nothing is created or modified; the discount
   just ends. This call is **free** (reservation describes + Savings Plan
   inventory); it flags both:
   - **Reservations** expiring near `--near` (`expiring_near_date`, marked
     `"expired": true`). A flagged reservation whose `type` matches the spiking
     usage-type/instance-class (e.g. expired `db.r7g.4xlarge` ⟷ spiking
     `InstanceUsage:db.r7g.4xl`) is the root cause at **high confidence** when the
     spike day == expiry day + 1 and instance-hours are flat.
   - **Savings Plans** that retired near `--near` (`savings_plans_expiring_near`
     + `savings_plan_lapse_hint`). **Check this whenever the spike is in compute
     and no RI expired** — a `retired` Compute/EC2 SP whose `commitment_per_hr`
     ≈ the daily on-demand $ jump (from step 2/3) is the confirmed root cause.
     This is the evidence a coverage % alone cannot give you: the CE coverage
     view can say "coverage is short" but only the inventory names *which* plan
     lapsed.
   - Add `--savings-plans --start <anomaly-start> --end <anomaly-end>` to *also*
     pull Savings Plans utilization and coverage from Cost Explorer. **Low
     `coverage_pct` + high `on_demand_cost_usd`** means a Savings Plan or new RI
     would recover the discount; size the commitment to the *steady-state*
     on-demand run-rate (use the baseline daily $ from step 2 / 3), not the peak.
     Only this CE view costs money (~$0.02); the inventory/flagging above is free.

6. **Write the report**
   Save to `reports/<anomaly-id-or-daterange>.md` in the user's working
   directory using the template below (create `reports/` if missing), then give
   the user a concise terminal summary: root cause in one or two sentences, the
   dollar impact, and the top remediation step.

## Analysis rules

- **Quantify every claim** with numbers from script output — dollar deltas,
  percentages, dates. Never estimate figures the scripts can give you.
- **Distinguish cost driver from root cause.** "EC2 data transfer rose $400"
  is the driver; "a NAT gateway was created on 06-03 by user X and traffic now
  routes through it" is the root cause.
- **Check CloudTrail before concluding.** A cost change with a matching
  configuration change is high confidence; without one it stays a hypothesis.
- **Flat usage + step-up cost = rate/coverage change, not new capacity.** If the
  instance-hours / storage / IO usage types are flat but cost jumps as a clean
  step, suspect a coverage lapse or a price change — run the step-5 reservation
  check before reaching for a CloudTrail event that may not exist. Note: under
  `UnblendedCost` (what `cost_breakdown.py` returns), an RI-covered line item
  reads as ~$0, so a lapse *looks like* a brand-new instance with a huge
  percentage jump — don't be fooled into reporting "new capacity."
- **A coverage lapse can be an RI *or* a Savings Plan.** When no RI expired near
  the spike, check the Savings Plan inventory (step 5,
  `savings_plans_expiring_near`) for a `retired` Compute/EC2 plan whose `end` ≈
  spike-day−1 and whose `commitment_per_hr` ≈ the daily on-demand $ increase —
  that is the confirming evidence the Cost Explorer coverage % alone cannot give.
  Don't conclude "no coverage lapse" from the RI list alone.
- **`cost_breakdown.py` cannot group by `RECORD_TYPE` or `PURCHASE_TYPE`.** To
  confirm a coverage lapse, don't reach for those dimensions — use
  `reservation_coverage.py` (reservation expiry dates + Savings Plan inventory)
  instead.
- **State confidence** (high / medium / low) and what evidence is missing.
- If CloudTrail returns nothing relevant, say so explicitly — the cause may be
  a reservation/Savings Plan lapse (step 5), organic usage growth, a pricing
  change, or a change older than the window. Do not invent a change event.
- If costs are spread thinly across many services with no clear spike, say
  that too; not every alert has a single root cause.

## Report template

```markdown
# Cost Anomaly Report — <anomaly id or date range>

## Summary
One paragraph: what spiked, by how much, when, and the most probable cause.

## Anomaly details
| | |
|---|---|
| Period | ... |
| Total impact | $... |
| Expected spend | $... |
| Actual spend | $... |
| AWS root-cause hint | ... |

## Root cause (confidence: high/medium/low)
The explanation, with the supporting chain: service → usage type → region →
triggering change (CloudTrail event + user + time, OR a reservation/Savings
Plan that expired on date X leaving the same usage to bill on-demand).

## Evidence
Tables of the relevant deltas (service-level, then drill-down) and the
relevant CloudTrail events.

## Timeline
Chronological list: baseline level → triggering change → spike start → (end/ongoing).

## Remediation
Concrete next steps, ordered by impact (e.g. delete/resize resource, add
lifecycle rule, set a budget alert, ask user X about intent). If the cause was
a coverage lapse: repurchase a matching RI or a Savings Plan sized to the
steady-state on-demand run-rate, and set a reservation-expiry alert so the
renewal does not lapse again.
```