# Soft Delete Follow-up: DELETED_HAS_ACCESS_TO ## Created Tickets: - https://theorchard.atlassian.net/browse/PP-1486 - `[lambda-pp-cdc] DELETED_HAS_ACCESS_TO Handler` - https://theorchard.atlassian.net/browse/PP-1485 - `[ows-pdp] Lambda Machine Identity Cerbos Policy` ## The Problem The existing lambda handles role changes by reacting to `operation: UPDATE` on the `LabelProfile` node, where a smaller (or empty) `roles` array drives `_diff_roles`. However, `ows-permissions` has 2 **soft-delete paths** ( `revoke-all-access-to-single_tenant`, `revoke-all-access`) that does not modify the `roles` array at all — it only creates a `DELETED_HAS_ACCESS_TO` relationship. The lambda never sees a role change and never detaches workstation roles in `ows-pdp`. - The `DELETED_HAS_ACCESS_TO` CDC topic is separate from the `Profile` update topic. - The CDC message contains the **Profile UUID and Vendor/Label UUID**, but **no roles array**. - A symmetrical path exists when access is granted again after revocation: ows-permissions assigns new roles and replaces the soft-deleted edge with a normal `HAS_ACCESS_TO`. ## Two New Events to Handle | Event | CDC operation | Lambda action | |---------------|------------------------------------|-------------------------------------------------------------------| | Soft delete | `DELETED_HAS_ACCESS_TO` **CREATE** | Resolve identity+vendor via ows-permissions; blindly detach all known workstation roles in `ows-pdp` | | Access re-grant | `DELETED_HAS_ACCESS_TO` **DELETE** | Attach workstation roles for this profile+tenant in `ows-pdp` | ## The Missing Roles Problem For **soft delete (CREATE)**: the lambda must detach roles, but the CDC message carries no roles. Options: - **Query `ows-pdp`** for roles currently attached to this profile — ground truth of what needs to be undone, but adds an extra round trip. - **Query `ows-permissions`** (Neo4j) for the `LabelProfile.roles` array before detaching. - **Detach all known workstation roles** blindly — simple; ows-pdp treats detaching a non-existent role as a no-op, so this is safe. The chosen approach is **detaching all known workstation roles blindly** via `attach_detach_roles_by_identity_tenant`. The lambda already calls `POST /lookup/profiles/identity/uuids/` on ows-permissions to resolve `identity_uuid` and `vendor_uuid`, so no additional round trip to ows-pdp is needed. For **access re-granted after revocation (DELETE)**: `_diff_roles` cannot be used here. It compares the CDC event's `before` vs `after` states — reflecting what changed in Neo4j, not what is missing in `ows-pdp`. Because ows-permissions never cleared the `roles` array during the soft delete, the `LabelProfile` UPDATE when access is re-granted may show a small diff or no diff at all, while `ows-pdp` has zero roles for that profile+tenant. The lambda must attach **all current roles**, not just the delta. ## Why Full Sync, Not Diff A `LabelProfile` UPDATE also fires when access is re-granted after revocation (confirmed empirically). When the same roles are granted again, the UPDATE produces an **empty diff** — the existing lambda no-ops and does not restore the detached roles. The `DELETED_HAS_ACCESS_TO` DELETE is the only actionable signal. When access is re-granted with **different roles**, the Profile UPDATE applies only the delta between before and after. This misses any roles that were present before the soft delete but absent from the diff — ows-pdp is empty and the delta alone does not restore them. In both cases the correct fix is the same: handle `DELETED_HAS_ACCESS_TO` **DELETE** by calling `POST /lookup/profiles/identity/uuids/` on ows-permissions to get the current roles and attaching all of them. This full sync is correct regardless of what roles changed and regardless of event ordering. Both the soft delete and access re-grant paths require new lambda logic. ## Scoping Note The `deletedBy` field in the example message (`revoke-all-access-to-single_tenant`) confirms the soft delete is * *tenant-scoped**. The profile may still hold roles for other tenants. Any detach must be scoped to the specific `(profile, tenant)` pair from the CDC event — not a global detach of all roles for the profile. ```json { "event": { "type": "DELETED_HAS_ACCESS_TO", "operation": "CREATE", "start": { "labels": [ "Profile" ], "keys": { "uuid": "38bdd41d-..." } }, "end": { "labels": [ "Orchard", "Label", "Vendor" ], "keys": { "uuid": "722a33a9-..." } } } } ``` The `start` node is the **Profile** (user), the `end` node is the **Vendor/Label** (tenant). Both UUIDs are present. ## Empirical Findings CDC events were captured for three operations against a test user with `Analytics (Workstation)` access to Vendor 7123. See `attach_detach_CDC_events/` for the raw events. **Operation 1 — Add access** `cdc.musicGraphV5.profile` fires with `roles: ["catalog", "analytics"]`. Expected. **Operation 2 — Revoke access (soft delete)** `cdc.musicGraphV5.deletedHasAccessTo` fires with `operation: "CREATE"` — as expected. But the **Profile node also fires a `cdc.musicGraphV5.profile` UPDATE** with the same `lastModifiedAt`. The `Profile.roles` array is unchanged: still `["catalog", "analytics"]`. - Both topics fire in the same transaction. - The Profile UPDATE produces an **empty `_diff_roles`** (no role change) — the existing lambda correctly no-ops. - The `DELETED_HAS_ACCESS_TO` CREATE is the only signal that workstation roles should be detached. **Operation 3 — Grant identical access again after revocation** `cdc.musicGraphV5.deletedHasAccessTo` fires with `operation: "DELETE"` — as expected. The **Profile node also fires a `cdc.musicGraphV5.profile` UPDATE** with the same `lastModifiedAt`. - Both topics fire in the same transaction. - Because the same roles were granted again, the Profile UPDATE produces an **empty `_diff_roles`** — the existing lambda no-ops. - The `DELETED_HAS_ACCESS_TO` DELETE is the only signal that workstation roles should be attached. **Implication for the design** The `DELETED_HAS_ACCESS_TO` DELETE handler is the sole path when access is re-granted after revocation — the Profile UPDATE cannot be relied on to drive the attach (the diff is empty when roles are unchanged). This validates the full-sync design in PP-1486. Also notable: `analytics` appears in `Profile.roles` but is not in `KNOWN_ROLES = {"administrator", "catalog"}`, so the lambda correctly ignores it in both the soft delete and access re-grant paths. ## Open Questions 1. ~~**Does `ows-pdp` expose an endpoint to list currently-attached roles for a `(profile_uuid, vendor_uuid)` pair?**~~ — **Resolved**: `GET /{identity_uuid}/roles/` ([identity.py:333](../../../ows-pdp/pdp/fastapi/routers/identity.py#L333)) returns all tenants and their roles for an identity. The lambda filters the response by `vendor_uuid` client-side. The endpoint calls `filter_for_identity_tenants`, which filters to tenants where the JWT identity and the queried identity share membership. The lambda's machine identity must therefore be a member of **both parent companies**: | Parent company | UUID | |----------------|----------------------------------------| | sme | `f1594122-7f99-4916-b103-08b0444c7b46` | | theorchard | `955a1bbd-b623-4ea1-ab5f-8d6620c442fb` | The machine policy needs two permissions on the `identity` resource (see `identity.yml`): - `list_tenants` — granted to `pde_team_pp` via `list_tenants_for_identity` (line 31), no self-restriction. - `attach_and_detach_role` — granted to `pde_team_pp` via `attach_and_detach_for_single_tenant` (line 56), conditioned on `P.id != R.attr.identity_uuid` (always satisfied since the lambda acts on other users). The self-only `list_tenants` pattern in `pdp_integration_test_machine.yml` is not sufficient. A new machine policy under `cerbos/policies/machines/pp/` is required. 2. ~~**Can we confirm that a `DELETED_HAS_ACCESS_TO` DELETE event and a `LabelProfile` UPDATE event are always emitted in the same transaction when access is re-granted after revocation? If so, can we rely on ordering?**~~ — **Resolved**: No, regardless of transaction boundaries. Both events land on different Kafka topics (`cdc.musicGraphV5.deletedHasAccessTo` vs `cdc.musicGraphV5.profile`) and are processed in separate Lambda invocations triggered independently by each topic's partition. MSK and Lambda's event source mapping provide no cross-topic ordering guarantee. The access re-grant handler must always do a full sync from ows-permissions. 3. ~~**Are there cases where a `DELETED_HAS_ACCESS_TO` CREATE fires but the `LabelProfile.roles` array is also cleared in the same transaction?**~~ — **Resolved** (updated): Both `cdc.musicGraphV5.deletedHasAccessTo` CREATE and a `cdc.musicGraphV5.profile` UPDATE fire in the same transaction during a soft delete — confirmed empirically. However, the Profile UPDATE carries the **same `roles` array as before** (soft delete does not touch roles), so `_diff_roles` produces an empty diff and the existing lambda no-ops. The `DELETED_HAS_ACCESS_TO` CREATE remains the sole actionable signal.