# SPIKE: How do we want to hydrate Feature Control data for an account or subaccount?

## Background

Cerbos resource policies can reference `account_feature_controls` — a list of feature IDs enabled
for a vendor (account) — in order to gate certain actions. For example, a policy might only allow
`bulk_create` if feature 43 is present in the tenant's feature controls list.

Feature controls are vendor-level only. They are stored in the `vendor_restricted_features` table
in `art_relations` (MySQL) and managed by ows-account. For a **subaccount** tenant, features are
inherited from the parent vendor; for an **account** tenant, they apply directly.

PDP (ows-pdp) currently has no mechanism for fetching or injecting this data before calling
Cerbos. This spike proposes two options.

---

## Option 1 — Fetch flag: add `VENDOR_FEATURES` to the existing lookup fetch flags in ows-account

### How it works

The existing `/lookup/vendors/uuids/`, `/lookup/vendors/vendor-ids/`,
`/lookup/subaccounts/uuids/`, and `/lookup/subaccounts/subaccount-ids/` endpoints in ows-account
already support a `fetch_flags` parameter. Today the only flag is `TENANT_HIERARCHY`, which causes
the response to include `company_brand_uuid` and `parent_company_uuid`.

We add a new flag: `VENDOR_FEATURES`. When this flag is present, ows-account enriches the
response with the list of enabled feature IDs for the vendor. For **subaccount** lookups,
ows-account resolves the parent vendor internally before fetching features — no PDP-side resolution
needed.

PDP's `OwsAccountClient` already passes `fetch_flags` to these endpoints. `MultiTenantProxy`
already calls them during hierarchy hydration. With this approach, we request features alongside
hierarchy in the same call by adding `VENDOR_FEATURES` to the flags passed by PDP.

On the PDP side, the response models (`LookupVendor`, `LookupSubaccount`) would gain an optional
`feature_ids` field. `MultiTenantProxy` writes this into
`resource.attributes.tenant.account_feature_controls` alongside the hierarchy data.

### Pros

- No new ows-account endpoint: extends an existing, well-understood mechanism.
- Subaccount→vendor resolution is handled inside ows-account, keeping PDP simple.
- Features and hierarchy travel in the same HTTP response — one fewer round-trip versus Options 1
  and 3 (where hierarchy and features are separate calls).
- Fetch flags are already optional, so the change is backwards compatible: callers that don't pass
  `VENDOR_FEATURES` are unaffected.
- Can be combined with Option 2's policy-driven approach: PDP only adds the `VENDOR_FEATURES` flag
  for resource types that need it.

### Cons

- Response payload is larger when the flag is used (feature IDs are included alongside hierarchy).
- Changes are needed in both services: ows-account (new flag, enriched response) and ows-pdp
  (new flag constant, updated response models, write features into attributes).
- `LOOKUP_FETCH_FLAGS` enum in ows-account currently only has `TENANT_HIERARCHY`; extending it is
  straightforward but does touch a shared constant.

### Required changes

**ows-account**:
- Add `VENDOR_FEATURES` to `LOOKUP_FETCH_FLAGS` in `account/constants/constants.py`.
- Update the four lookup handlers to join against `vendor_restricted_features` when the flag is
  present and include feature IDs in the response.
- For subaccount lookups with `VENDOR_FEATURES`, resolve `vendor_id` from the subaccount record
  before fetching features.

**ows-pdp**:
- Add `VENDOR_FEATURES` to `LookupVendorFetchFlags` enum in `pdp/connectors/ows_account.py`.
- Add optional `feature_ids: Optional[List[int]]` to `LookupVendor` and `LookupSubaccount` models.
- `MultiTenantProxy` passes `VENDOR_FEATURES` flag (conditionally or always) and writes feature IDs
  into resource attributes after hierarchy hydration.

### Key diffs

**`account/constants/constants.py`**
```python
# before
LOOKUP_FETCH_FLAGS = Enum('FETCH_FLAGS', [FETCH_TENANT_HIERARCHY])

# after
FETCH_VENDOR_FEATURES = 'VENDOR_FEATURES'
LOOKUP_FETCH_FLAGS = Enum('FETCH_FLAGS', [FETCH_TENANT_HIERARCHY, FETCH_VENDOR_FEATURES])
```

**`pdp/connectors/ows_account.py`**
```python
class LookupVendorFetchFlags(str, Enum):
    TENANT_HIERARCHY = "TENANT_HIERARCHY"
    VENDOR_FEATURES = "VENDOR_FEATURES"           # new

class LookupVendor(BaseModel):
    vendor_id: int
    uuid: UUID
    company_brand_uuid: Optional[UUID] = None
    parent_company_uuid: Optional[UUID] = None
    feature_ids: Optional[List[int]] = None       # new

class LookupSubaccount(BaseModel):
    subaccount_id: int
    uuid: UUID
    vendor_uuid: Optional[UUID] = None
    company_brand_uuid: Optional[UUID] = None
    parent_company_uuid: Optional[UUID] = None
    feature_ids: Optional[List[int]] = None       # new
```

**`pdp/proxies/multi_tenant_proxy.py`** — pass the flag and write feature IDs alongside hierarchy
```python
# in _get_account_tenant_hierarchies_from_ows_account
response = await self._ows_account_client.lookup_vendors_by_uuids(
    uuids=tenant_uuids,
    fetch_flags=[LookupVendorFetchFlags.TENANT_HIERARCHY, LookupVendorFetchFlags.VENDOR_FEATURES],
)

# after building tenant_hierarchies dict, also return feature_ids keyed by uuid
# (caller writes them into resource.attributes["tenant"]["account_feature_controls"])
```

---

## Option 2 — Policy-driven: parse Cerbos policies in CI to determine when to fetch

### How it works

A Jenkinsfile stage crawls the Cerbos YAML policy files **once per deployment** to build a
`PolicyMetadataDatabase` that maps each resource type to what data its policy requires.
Specifically, it detects whether a resource policy's `variables.import` references
`account_feature_controls`. The database is serialized to JSON and written to Redis under a
fixed key (`cerbos_policy_metadata`). This mirrors the existing `bludgeon_cached_cerbos_decisions`
CI pattern.

At startup each Fargate task reads the Redis entry, deserializes it into a
`PolicyMetadataDatabase` instance, and injects it via the existing datasources/DI pattern. No
task ever touches the filesystem.

Before each Cerbos call, for each resource in the request, the task consults the in-memory
database. Only fetch feature controls for resource types whose policy actually requires them.
Resources whose policies do not reference `account_feature_controls` are skipped entirely.

For **account** tenants: call `POST /lookup/vendors/features/uuids/` with the tenant UUID.
For **subaccount** tenants: call `POST /lookup/subaccount/features/uuids/` directly with the
subaccount UUID — ows-account resolves to the parent vendor internally and returns feature IDs
keyed by the original subaccount UUID.

### Pros

- Avoids unnecessary calls: feature data is only fetched when the policy actually uses it.
- Scales cleanly as new resource types are added — the policy file determines the behavior, not
  hand-maintained config.
- Policy parsing is a one-time CI cost; no per-task filesystem access at startup.
- Fargate tasks start fast — they only do a single Redis read to get the database.

### Cons

- More moving parts: policy parser, CLI command, Redis cache entry, Jenkinsfile stages.
- If a policy is deployed without the seed step running (e.g., the stage is skipped), tasks will
  use a stale database until the next seed (mitigated by always running seed alongside deploy).
- Requires two new ows-account endpoints (one for vendor UUIDs, one for subaccount UUIDs).

### Required changes

**ows-account**:
- New endpoint `POST /lookup/vendors/features/uuids/` — accepts vendor UUIDs, returns enabled
  feature IDs per vendor: `{"vendors": {"uuid1": [39, 43], ...}}`.
- New endpoint `POST /lookup/subaccount/features/uuids/` — accepts subaccount UUIDs, resolves
  to the parent vendor internally, returns enabled feature IDs keyed by subaccount UUID:
  `{"subaccounts": {"subaccount_uuid1": [39, 43], ...}}`.

**ows-pdp**:
- New `CerbosPolicyParser` + `PolicyMetadataDatabase` (with `to_json`/`from_json`) in
  `pdp/connectors/cerbos_policy_parser.py`.
- New `CACHE_ENTRY_CERBOS_POLICY_METADATA = "cerbos_policy_metadata"` constant in
  `pdp/constants/constants.py`.
- New Typer CLI command `seed_policy_metadata_cache` in `pdp/cli/commands/cerbos.py` —
  crawls the policy dir, builds the database, writes JSON to Redis.
- New `make seed_policy_metadata_cache` Makefile target.
- New Jenkinsfile stages `Seed QA Policy Metadata Cache` / `Seed Prod Policy Metadata Cache`,
  guarded by `isCerbosPolicyUpdated()`, placed after the corresponding `Bludgeon *Cache` stages.
- `datasources_lifespan` reads the Redis entry and deserializes on startup (fails loudly if missing).
- New hydration function `_hydrate_resources_with_feature_controls_as_needed` in `cerbos.py`.
- New `get_features_for_vendor_uuids` and `get_features_for_subaccount_uuids` methods on
  `OwsAccountClient`.
- `check_resources` calls the new hydration function before calling Cerbos.

### Key diffs

**`pdp/constants/constants.py`** — new cache key
```python
CACHE_ENTRY_CERBOS_POLICY_METADATA = "cerbos_policy_metadata"

class CacheEntryType(str, Enum):
    ...
    CACHE_ENTRY_CERBOS_POLICY_METADATA = CACHE_ENTRY_CERBOS_POLICY_METADATA
```

**`pdp/cli/commands/cerbos.py`** — CI seed command
```python
@cli.command("seed_policy_metadata_cache")
def seed_policy_metadata_cache(policies_dir: str = "cerbos/policies") -> None:
    """Crawl policy dir, build PolicyMetadataDatabase, write JSON to Redis."""
    asyncio.run(_seed_policy_metadata_cache(policies_dir))
```

**`Jenkinsfile`** — seed stage (QA shown; Prod is identical with prod vars)
```groovy
stage('Seed QA Policy Metadata Cache') {
    when {
        allOf {
            branch 'master'
            expression { isCerbosPolicyUpdated() == true }
        }
    }
    steps {
        withEcr {
            withEnv(["REDIS_URL=${QA_REDIS_URL}", "CACHE_USE_REDIS=${CACHE_USE_REDIS}"]) {
                withAWS(role: QA_DEPLOYMENT_ROLE, roleAccount: QA_ACCOUNT_ID, ...) {
                    sh 'make seed_policy_metadata_cache'
                }
            }
        }
    }
}
```

**`pdp/fastapi/datasources.py`** — read from Redis at startup (not from filesystem)
```python
raw = await redis_connector.get(key=CACHE_ENTRY_CERBOS_POLICY_METADATA)
if raw is None:
    raise RuntimeError("Cache entry 'cerbos_policy_metadata' not found. Run seed step.")
policy_metadata_db = PolicyMetadataDatabase.from_json(raw)
DATA_SOURCES[POLICY_METADATA_DATABASE_KEY] = policy_metadata_db
```

**`pdp/connectors/ows_account.py`** — new response models and client methods
```python
class LookupVendorFeaturesResponse(BaseModel):
    vendors: Dict[str, List[int]]  # vendor_uuid -> [feature_id, ...]

class LookupSubaccountFeaturesResponse(BaseModel):
    subaccounts: Dict[str, List[int]]  # subaccount_uuid -> [feature_id, ...]

async def get_features_for_vendor_uuids(self, uuids: List[UUID]) -> LookupVendorFeaturesResponse:
    response = await self.async_ows_client.post(
        self.service_name,
        path="/lookup/vendors/features/uuids/",
        json={"uuids": [str(u) for u in uuids]},
    )
    response.raise_for_status()
    return LookupVendorFeaturesResponse.model_validate_json(response.content)

async def get_features_for_subaccount_uuids(
        self, uuids: List[UUID]
) -> LookupSubaccountFeaturesResponse:
    response = await self.async_ows_client.post(
        self.service_name,
        path="/lookup/subaccount/features/uuids/",
        json={"uuids": [str(u) for u in uuids]},
    )
    response.raise_for_status()
    return LookupSubaccountFeaturesResponse.model_validate_json(response.content)
```

**`pdp/logic/cerbos.py`** — new hydration step, inserted before the Cerbos call
```python
# Account and subaccount UUIDs are sent to their respective dedicated endpoints;
# ows-account handles vendor resolution for subaccounts internally.
check_resources_request = await _hydrate_resources_with_feature_controls_as_needed(
    check_resources_request=check_resources_request,
    policy_metadata_db=policy_metadata_db,
    ows_account_client=ows_account_client,
)
```

**`pdp/connectors/cerbos_policy_parser.py`** — parser and serializable database
```python
class PolicyMetadataDatabase:
    def requires_account_feature_controls(self, resource_type: str) -> bool:
        policy = self._db.get(resource_type)
        return policy.requires_account_feature_controls if policy else False

    def to_json(self) -> str: ...      # serialize for Redis write (CI)
    @classmethod
    def from_json(cls, raw: str) -> "PolicyMetadataDatabase": ...  # deserialize at startup

    @classmethod
    def build_from_policy_dir(cls, policy_dir: Path) -> "PolicyMetadataDatabase":
        # parse each *.yaml in policy_dir, check variables.import for
        # "account_feature_controls" and importDerivedRoles for "workstation_roles"
        ...
```

---

## Comparison

| | Option 1 (Fetch flag) | Option 2 (Policy-driven) |
|---|---|---|
| New ows-account endpoint | No | Yes (2 endpoints) |
| Extra HTTP call for features | No (same call as hierarchy) | Yes (separate) |
| Subaccount resolution | In ows-account | In ows-account |
| Fetches only when needed | Depends on how PDP passes the flag | Yes |
| Proxy class changes | Minimal | None |
| Policy DB build cost | N/A | Once in CI; startup reads Redis |
| Implementation complexity | Medium | High |
| ows-account changes | Low-medium | Medium |
| ows-pdp changes | Medium | High |
| Jenkinsfile/CI changes | None | Yes (seed stages for QA + Prod) |