# Claude Query Comparison Analysis

Analysis of `orchardLabels`/`orchardLabelSearch` vs `accountsxPP` query results for PP-1260.

Note: Both queries return signed vendors only — `orchardLabelSearch` filters internally to `status: "SIGNED"`, and `accountsxPP` is filtered to `status: ["SIGNED"]`.

## Executive Summary

Significant divergence between the two query approaches, with **44 of 104 searches (42.3%) returning the same first result**. The default account lists (no search term) have **zero overlap**. Overlap at the result-set level has improved versus the unfiltered comparison (85.6% of searches now share at least one account), but ordering rarely matches.

## Overall Statistics

- **Total queries analyzed**: 104
  - 1 no-term query (orchardLabels vs accountsxPP)
  - 103 search term queries (orchardLabelSearch vs accountsxPP)

### Match Quality

| Metric | Results |
|--------|---------|
| **First result matches** | 44 (42.3%) |
| **Perfect matches** (all results identical) | 0 (0.0%) |
| **1+ matching results** | 89 (85.6%) |
| **3+ matching results** | 23 (22.1%) |
| **5+ matching results** | 15 (14.4%) |
| **7+ matching results** | 2 (1.9%) |

### Result Availability

- **Zero orchard results**: 4 queries (terms: "25359", "32211", "66289", "Animaj Investment SPV SAS")
- **Zero accountsxPP results**: 0 queries
- **No overlap (but both have results)**: 11 (10.6%)

## Key Findings

### 1. No-Term Query Shows Complete Divergence

The comparison of default account lists (first 10 accounts with no search term) shows:
- **First result match**: No
- **Matching accounts**: 0 out of 10

This is because the `orchardLabels` [query](https://github.com/theorchard/graphql-knowledge/blob/ec9e900bef774347ba712da93607418f596c9f63/src/connectors/neo4j/Label/cypher/list.cypher#L91) is ordered by character ascending — orchard returns names starting with `!`, `"`, `#`, `$`, while xPP returns a different default ordering entirely.

### 2. Number Searches Outperform Text Searches

Analyzing first result matches by search type:

| Search Type | Total | First Match | Match Rate |
|-------------|-------|-------------|------------|
| **Number searches** (account IDs) | 27 | 17 | 63.0% |
| **Text searches** (account names) | 76 | 27 | 35.5% |

Number searches still show meaningfully higher first-result agreement than text searches, but both rates are lower than in the prior unfiltered comparison.

### 3. Very Few High-Overlap Matches

Only **2 (1.9%)** had 7+ matching results:
- "RIMAS Entertainment LLC" — 8/10 matches, first result matches
- "rimas" — 9/10 matches, but first result differs (xPP ranks "Rimas Entertainment LLC d/b/a Rimas Mexico" above the exact-name match)

Several terms that previously cleared the 7+ threshold ("KMR", "concord", "Columbia") now sit between 5 and 6 matches, indicating tighter divergence under the SIGNED-only filter.

### 4. Common Pattern: First Match But Then Diverges

**44 (42.3%)** match on the first result. Analyzing these cases:

| Scenario | Count | Percentage |
|----------|-------|------------|
| **Orchard returns only 1 result** | 38 | 86.4% |
| **All orchard results match, xPP has more** | 5 | 11.4% |
| **Both return multiple, but diverge** | 1 | 2.3% |

The "first match" outcome is overwhelmingly driven by the orchard-returns-one case: xPP almost always ranks that exact match first, but then adds many fuzzy results below it.

**Examples where orchard returns only 1**:
- "18686", "25153", "25824" — orchard finds the exact match; xPP returns it plus 9 additional results

**Examples where all orchard results match, xPP has more**:
- "Membran" — orchard: 6 results, xPP: 10, all 6 orchard results appear in xPP and first matches
- "Santa Anna" — orchard: 6 results, xPP: 10, all 6 match with first matching
- "thirty tigers" — orchard: 2, xPP: 10, both match
- "RCA Records" — orchard: 3, xPP: 10, all 3 match with first matching

**Example of genuine divergence**:
- "RIMAS Entertainment LLC" — orchard: 8, xPP: 10, 8 match with first matching

### 5. No Overlap Cases

**10 search-term queries** (plus the no-term query) return results from both systems but have **zero matching accounts** (when comparing by vendorId + name):

`794`, `794753`, `794787`, `carlos`, `LOFI Publishing Sasu`, `niti`, `one`, `Rebl`, `SME`, `ste`

The previously-zero-overlap terms `Epic`, `french`, and `HANDS` now share at least one result with orchard, suggesting the SIGNED-only filter has narrowed xPP's noise in some cases.

Several of the remaining no-overlap cases are data-quality artefacts rather than true search-quality failures — see Finding 7.

### 6. Cases Where accountsxPP Returns Results But orchard Returns None

**4 cases** where `orchardLabelSearch` returns 0 results but `accountsxPP` returns 10:
- `25359`, `32211`, `66289` — numeric IDs
- `Animaj Investment SPV SAS` — exact-name search that xPP nonetheless surfaces fuzzy hits for

This indicates `accountsxPP` has broader matching capabilities for certain search patterns, though for the user-facing flow it can mean a result list with no relevant entries.

### 7. Pattern: Exact Match vs Fuzzy Search

Analyzing cases where orchard returns exactly 1 result (51 cases) and examining whether xPP includes it:

- **38 cases (74.5%)** — xPP ranks the exact match first
- **9 cases** — xPP includes the exact match, but not first:
  `790717`, `793446`, `Animaj`, `engine comp`, `feede`, `hasbro`, `naika`, `SME Mexico, S.A. de C.V.`, `SME US LATIN LLC`
- **4 cases** — orchard's exact match is missing from xPP's `(vendorId, name)` tuples:
  `794`, `794753`, `794787`, `LOFI Publishing Sasu`

The 4 "missing" cases all turn out to be data-quality artefacts where the underlying `vendorId` does exist in xPP, but the `name` field doesn't match:

- **`794`** — orchard returns vendorId `794` "Celtic Whispers"; xPP returns vendorId `794` with an empty name (along with 9 unrelated fuzzy results)
- **`794753`** and **`LOFI Publishing Sasu`** — orchard returns vendorId `794753` "LOFI Publishing Sasu " (trailing space); xPP returns the same vendorId multiple times under cypress E2E test names (`cypress_e2e_abacus_account_*`), with one entry as "LOFI Publishing Sasu" *without* the trailing space — so the tuple match fails purely on whitespace
- **`794787`** — orchard returns vendorId `794787` "Santa Anna"; xPP returns the same vendorId under cypress/playwright E2E test names

These four cases are QA-data noise in the test environment, not production search-quality failures.

### 8. Relevance Ranking Observations

**Cases where xPP ranks relevant results lower**:
- `bolero` — orchard: 2 results (Bolero Records, Billy Bolero Entertainment AB); xPP returns "Kreativ Bokerød" first, with both bolero results immediately after
- `hasbro` — orchard: 1 (Hasbro Consumer Products Licensing Ltd.); xPP returns "Harbron, Robert" first, with Hasbro at position 2
- `naika` — orchard: 1 (Naika LLC); xPP returns "Laxman Naikwadi" first (and three times overall), with Naika LLC at position 2
- `Animaj` — orchard: 1 (Animaj Studios); xPP places it at position 10 behind nine "Animal*" entries
- `feede` — orchard: 1 (Feedelity / Hans-Peter Lindstrøm); xPP returns "Feeda" first, with Feedelity at position 2

**Cases with different matching strategies**:
- `kmc` — orchard surfaces fuzzy person matches ("Jack McAllister", "Jack McEwan", etc.) alongside KMC labels; xPP leads with "KMC Music Limited - GBP" and is more label-focused, but adds noise like "KMI (SMC-Private) Limited"
- `KMR` — orchard: 8 results, xPP: 10, 5 overlap; orchard leads with "A5B, Inc. d/b/a Lauv KMR II", xPP leads with "KMR Music Royalties II SCSp,"
- `rimas` — both return 10 results with 9 overlapping (highest overlap of any term), but xPP ranks "Rimas Entertainment LLC d/b/a Rimas Mexico" above the more obvious "RIMAS Entertainment LLC" exact name match

## Recommendations

1. **Investigate default account logic**: The no-term query returns completely different account sets between the two systems.

2. **Analyze text search ranking**: Text searches show only a 35.5% first-match rate compared to 63.0% for number searches — ranking parity is the largest UX risk.

3. **Review missing matches**: 4 search terms return results in xPP but not in orchard (3 numeric, 1 long-form name).

4. **Examine relevance ranking**: Cases like `bolero`, `hasbro`, `naika`, and `Animaj` demonstrate that xPP often *contains* the right answer but ranks it below fuzzier matches.

5. **Investigate data hygiene in QA**: Trailing-space names and cypress/playwright E2E account names appear under the same vendorId in xPP (`794753`, `794787`), which both inflates result counts and breaks tuple-level matching. Confirm whether this is QA-only or also occurs in production.

## Next Steps

- [ ] Investigate why default accounts differ completely
- [ ] Manual testing of high-divergence cases with product team
- [ ] Decision on acceptable divergence threshold
- [ ] Confirm whether the duplicate-vendorId / E2E-account-name noise is isolated to QA

---

_Analysis generated from 104 query comparisons | accountsxPP filtered to signed vendors only_