# D2C Shopify Data Connector Onboarding — Terraform

This directory contains the EventBridge/Lambda infrastructure for automating Shopify store onboarding through Fivetran into Snowflake.

## Resource Naming

D2C resources in the ecommerce accounts use the `d2c-` prefix:

- DynamoDB tables: `d2c-*`
- S3 buckets: `d2c-*`
- IAM roles: `d2c-*` (e.g. `d2c-role`)
- Lambda functions: `lambda-d2c-*`

The D2C application family is `d2c`. Team name tag is `d2c`.

## Architecture

Three EventBridge/SQS-triggered Lambda functions:

1. **lambda-d2c-connect-shopify-stores** (hourly) — Scans ARTIST_STORES, enqueues new stores to SQS
2. **lambda-d2c-onboard-shopify-stores** (SQS consumer) — Creates Fivetran connectors, issues Connect Cards
3. **lambda-d2c-sync-shopify-data-connectors** (every 15 min) — Advances connector state, upserts registry

State machine: `pending_oauth → pending_configure → pending_sync → sync_complete`

## Deployment Safety

Triggers are controlled by `eventbridge_schedule_enabled` (applies to all three Lambdas atomically). It defaulted to `false` until the required Secrets Manager values were populated:

- `prod/shopify-data-connector-onboarding/SNOWFLAKE_PRIVATE_KEY` — RSA private key PEM
- `prod/shopify-data-connector-onboarding/FIVETRAN_CREDENTIALS` — JSON with `api_key`, `api_secret`

Those secrets are now populated, so the default is `true`.

## Critical Invariants

- **FIVETRAN_GROUP_ID is validated at startup** — must be `stimuli_armrest` (prod destination)
- **Schema name comes from connector, not state** — use `conn.name` for SHOP table lookups
- **Enqueue before write** — Lambda 1 sends to SQS before writing registry row (allows safe recovery)
- **Respect invocation budget** — Lambda 3 stops processing when < 120s remains, defers stores to next sweep

## Related

- Snowflake service user: `PROD_LAMBDA_D2C_SHOPIFY_ONBOARDING` (separate PR, must be applied first)
- Snowflake schema/tables: `SHOPIFY_STORES_GLOBAL.D2C_OPERATIONS` (database PR)
- Python Lambda code: `/Users/cbeesley/d2c/bulk-shopify-onboarding/lambda/`