## extract-native-configuration

AWS Lambda that connects to an Aurora (MySQL or PostgreSQL) cluster, safely resets master credentials, extracts a schema‑only logical definition of all user databases, captures any user‑modified cluster parameter group settings, and uploads the resulting artifacts to S3 for downstream analysis or migration tooling.

### Key Capabilities
* Supports Aurora MySQL and Aurora PostgreSQL
* Resets (rotates) master credentials prior to extraction (via common shared library)
* Produces a schema‑only dump (no data, roles, or passwords)
  * MySQL: `mysqldump --no-data --routines --events --triggers --single-transaction`
  * PostgreSQL: iterates user databases with `pg_dump --schema-only --no-owner --no-privileges --no-comments` (one per DB) and concatenates results
* Filters out system / template databases (see `config.OMITTED_DATABASES`)
* Captures engine + major.minor version to `engine.json`
* Writes user‑modified parameter overrides (excluding omitted prefixes) to `my.cnf` or `postgresql.conf`
* Uploads all artifacts to S3 under `artifacts/<cluster_identifier>/`
* Sentry integration (optional) for error tracking

### Generated S3 Objects
```
artifacts/<db_name>/schema.sql          # Combined schema only DDL
artifacts/<db_name>/engine.json         # {"engine": "mysql|postgresql", "version": "X.Y"}
artifacts/<db_name>/my.cnf              # (only if user parameters & MySQL)
artifacts/<db_name>/postgresql.conf     # (only if user parameters & PostgreSQL)
```

### Event Payload
```json
{
  "db_name": "my-aurora-cluster-id",
  "db_type": "cluster"
}
```
`db_type` is currently expected to be `cluster` (aligns with existing common DB utility functions).

---
## Local Development

### Prerequisites
* Docker + Docker Compose
* (Optional) Access to AWS credentials if invoking against real infrastructure (tests use moto mocks)

### Start Runtime Container
```
docker compose up --build -d function
```

### Invoke Locally
```
curl -X POST \
  http://localhost:9000/2015-03-31/functions/function/invocations \
  -H 'Content-Type: application/json' \
  -d '{"db_name": "example-cluster", "db_type": "cluster"}'
```

### Live Code Reload
`src/` and `config.py` are bind‑mounted; rebuild is not required for Python changes. Restart container to reload:
```
docker compose restart function
```

---
## Configuration

| Variable | Purpose | Default |
|----------|---------|---------|
| `AWS_REGION` | AWS region for API calls | `us-east-1` |
| `ENVIRONMENT` | Deployment environment label used by logging/Sentry | `dev` |
| `S3_BUCKET_NAME` | Destination bucket for artifacts | `dev-orcdbucket` |
| `PENDING_CHANGES_WAIT_TIMEOUT` | Seconds to wait for RDS pending changes when rotating credentials | `300` |
| `SENTRY_DSN` | Enables Sentry if provided | (unset) |
| `SERVICE_NAME` | Service identifier for logging/traces | `rds-utils-extract-native-configuration` |

Internal constants of interest:
* `OMITTED_DATABASES` – system/template DBs skipped per engine
* `OMITTED_PARAMETER_GROUP_PATTERNS` – parameter name prefixes to ignore (case-insensitive startswith)
* `PASSWORD_MIN_LENGTH` – enforced for generated credential resets

---
## How Schema Extraction Works
### MySQL
1. Filters databases not in `OMITTED_DATABASES["mysql"]`.
2. Runs a single `mysqldump` including routines/events/triggers schema only.

### PostgreSQL
1. Queries `pg_database` for non-template, non-system DBs.
2. Runs `pg_dump` once per database (schema only, no ownership/privileges/comments).
3. Concatenates each with a header line: `-- ===== Database: <name> =====`.

No user/role/password statements are included.

---
## Testing & Linting
### Run Tests + Lint
```
docker compose up --build lint-and-test
```

Adjust pytest arguments via `TEST_ARGS` in `docker-compose.yaml` (e.g. `-k postgresql -vv`).

Coverage:
* Change `COV_REPORT=html` (temporary) then browse http://localhost:8000/
* Use `COV_REPORT=off` to disable.

Skip lint temporarily (local only): set `SKIP_LINT=1` (do not commit intentionally unless agreed).

Capture exit code in CI‑like mode:
```
docker compose up --exit-code-from lint-and-test --abort-on-container-exit --build lint-and-test
```

---
## Error Monitoring (Sentry)
Set `SENTRY_DSN` and `ENVIRONMENT` to enable tracing of exceptions during Lambda execution.

---
## Datadog (Optional Pattern)
If wrapped by the Datadog Lambda extension/module in deployment, ensure environment variables:
* `ENVIRONMENT`
* `DD_LAMBDA_HANDLER=src.app.handler`

---
## Outputs Summary
| File | Description |
|------|-------------|
| `schema.sql` | Aggregated schema (per-db for PostgreSQL) |
| `engine.json` | Engine + major.minor version metadata |
| `my.cnf` / `postgresql.conf` | User modified parameter overrides |