# Luminate Data Overview

## Databases and Schemas

### LUMINATE_DB_LISTING_DETAIL.EXTRACT_S (Vendor Datashare — 41 views)

This is the primary datashare from Luminate. Contains all metadata, consumption facts, reference data, mapping data, chart data, and airplay data.

**Metadata views** (entity descriptors — not territory-specific):
- `VW_MUSICAL_RECORDING_DS` — track-level metadata (ISRCs)
- `VW_SONG_DS` — song-level metadata
- `VW_MUSICAL_PRODUCT_DS` — product-level metadata (barcodes)
- `VW_MUSICAL_RELEASE_DS` — release-level metadata
- `VW_MUSICAL_RELEASE_GROUP_DS` — album-level metadata
- `VW_ARTIST_DS` — artist metadata

**Consumption fact views** (territory-specific — see `consumption-facts.md` for detail):
- Detail views: `VW_DAILY_FACT_{ENTITY}_DETAIL_DS` — market-level granularity
- Summary views: `VW_DAILY_FACT_{ENTITY}_SUMMARY_DS` — national-level, faster
- Airplay views: `VW_DAILY_FACT_{ENTITY}_AIRPLAY_DETAIL_DS` (for Song, Artist, MRELG)
- Provider views: `VW_DAILY_FACT_{ENTITY}_PROVIDER_DS` — provider-level activity (requires participation)
- Analysis view: `VW_WEEKLY_FACT_ANALYSIS_DS` — industry-level weekly aggregates by market/country

**Mapping views** (entity-to-entity relationships — see `entity-relationships.md`):
- `VW_MR_SONG_MAP_DS` — Musical Recording ↔ Song
- `VW_MR_MP_MAP_DS` — Musical Recording ↔ Musical Product
- `VW_MP_MREL_MAP_DS` — Musical Product ↔ Musical Release
- `VW_MREL_MRELG_MAP_DS` — Musical Release ↔ Musical Release Group
- `VW_SONG_MRELG_MAP_DS` — Song ↔ Musical Release Group

**Reference views** (lookup/dimension data):
- `VW_COUNTRY_DS` — country codes and names
- `VW_MARKET_DS` — US/CA metropolitan markets
- `VW_DATE_DS` — time dimensions (chart weeks, months, quarters)
- `VW_GENRE_DS` — genre hierarchy (levels, Luminate vs Billboard definitions)
- `VW_FACT_VALUES_DS` — complete list of metric categories, breakouts, and breakout values
- `VW_DATA_SOURCES_DS` — data providers who reported activity per timeframe/territory
- `VW_RIGHT_HOLDER_DS` — rights holder information

**Chart views**:
- `VW_CHART_DS` — chart catalog (chart names, types, frequencies, entities)
- `VW_CHART_DATA_DS` — chart rankings per week (CHART_ID + WEEK_ID + ENTITY_ID + RANK)
- `VW_CHART_DATA_SOURCES_DS` — data sources contributing to charts

**Airplay reference views**:
- `VW_AIRPLAY_FORMAT_DS` — airplay format lookup
- `VW_STATION_DS` — airplay station details (call letters, frequency, zip)
- `VW_STATION_FORMAT_MAP_DS` — station ↔ format mapping (point-in-time, keyed by REPORT_DATE)
- `VW_SONG_AIRPLAY_FORMAT_DS` — song CRG (current/re-current/gold) status per format and week

**Market share view**:
- `VW_WEEKLY_FACT_MARKET_SHARE_DS` — weekly activity by business unit (Next Gen methodology)

### LUMINATE_DB_LISTING_CLAIM.EXTRACT_S (Vendor Datashare — Claims, 7 views)

Claim data is for content that is owned and/or distributed. Restricted to US and CA only. Consumers must participate in the claims process.

- `VW_DAILY_FACT_MR_CLAIM_DS` — track-level claim consumption (includes CONTENT_CREATION_CATEGORY: Official/UGC)
- `VW_DAILY_FACT_MP_CLAIM_DS` — product-level claim consumption
- `VW_BUSINESS_UNIT_DS` — business unit lookup
- `VW_BUSINESS_UNIT_HIERARCHY_DS` — business unit parent/child hierarchy
- `VW_CATALOG_CODE_DS` — catalog code lookup
- `VW_MUSICAL_RIGHT_DS` — musical rights metadata
- `VW_WEEKLY_FACT_MARKET_SHARE_LEGACY_DS` — legacy market share methodology

### LUMINATE_MODELS.PROD (Internal Aggregate Models)

Pre-aggregated tables built from Summary fact views. See `internal-models.md` for full details.

- Period-end models: `{PERIOD}_{ENTITY}_SUMMARY` where PERIOD is RTD, WTD, YTD, or RTCW
- Monthly models: `MONTHLY_{ENTITY}_SUMMARY` (Artist, MR, MP only)

## Entity Hierarchy

Luminate organizes music data around 6 entity types with a hierarchical relationship:

```
Artist
  └── Song
        └── Musical Recording (MR) — identified by ISRC
              └── Musical Product (MP) — identified by barcode/ICPN
                    └── Musical Release (MREL)
                          └── Musical Release Group (MRELG) — album level
```

**Key ID columns per entity:**
| Entity | ID Column | External ID | Common Name |
|--------|-----------|-------------|-------------|
| Artist | ARTIST_ID | — | Artist |
| Song | SONG_ID | — | Song |
| Musical Recording | MR_ID | ISRC | Track / ISRC |
| Musical Product | MP_ID | ICPN (barcode) | Product / Barcode |
| Musical Release | MREL_ID | — | Release |
| Musical Release Group | MRELG_ID | — | Album |

Note: The hierarchy is not strictly 1:1. A Song can have multiple Musical Recordings (different ISRCs for different versions). A Musical Recording can appear on multiple Musical Products (same track on different album editions).

## Data Refresh

- **Datashare views**: Updated nightly at 2:00am EST. No downtime during refresh.
- **Monthly internal models**: Refreshed on the 5th of every month with all historical data for completed months.
- **Period-end internal models**: Refreshed to reflect the current reporting period.
- **Timestamps**: `MODIFIED_AT` fields use UTC timezone.