# POC Overview: Heavy Rotation Listeners

**Initiative:** Listening Behavior Collector
**Date:** October 3, 2025

---

## 1. 📂 Project Background

[cite_start]Our company has an opportunity to gain deeper insights into fan engagement by leveraging the listening behavior data our fans have authorized us to access on Spotify[cite: 1]. [cite_start]The full **Listening Behavior Collector** initiative aims to build a system that can extract this data at a massive scale, starting with our current dataset of approximately 73 million fans, which is growing every year[cite: 1].

The primary business goal is to identify valuable fan segments, such as "Heavy Rotation" listeners for our artists. This will allow for more targeted marketing, enhanced fan engagement strategies, and a better understanding of how music is consumed.

This document outlines a small, fast-paced Proof of Concept (POC) to validate our technical approach before committing to the full-scale build.

---

## 2. 🎯 POC Objective

The objective of this POC is to **prove the technical feasibility** of our proposed data collection and analysis workflow. We aim to:

* Validate our ability to make authenticated calls to the Spotify API from within our Snowflake environment.
* Test an asynchronous, serverless job model using Snowflake Tasks to handle a batch data-fetching workload.
* Develop a simple, interactive Streamlit application to serve as a tangible demo for stakeholders.
* Gain initial performance benchmarks and learnings that will directly inform the architecture of the full-scale production system.

---

## 3. 🗺️ Scope

[cite_start]To ensure we can deliver this POC quickly and learn effectively, we are adhering to our principle of **fast iteration** by tightly scoping the work[cite: 2350]. This is a "Small Batch" project with a 1-2 week appetite.

#### ✅ In Scope:

* [cite_start]**Limited Dataset:** The POC will operate on a small, controlled dataset of **5 pre-selected artists** and **10,000 fan tokens per artist**[cite: 1].
* **Asynchronous Workflow:** The entire workflow will be asynchronous, triggered from a UI but executed in the background. [cite_start]This prevents the UI from timing out and simulates our final production design[cite: 1].
* **Technology Stack:** All components will be built within Snowflake, utilizing:
    * [cite_start]A **Streamlit in Snowflake** application for the user interface[cite: 1].
    * [cite_start]A **Python UDF** for making secure, external API calls to Spotify[cite: 1].
    * [cite_start]A **Snowflake Serverless Task** to orchestrate the data collection job[cite: 1].
* **Core Functionality:** The application will allow a user to select an artist, trigger the analysis, and view the aggregated count of "Heavy Rotation" listeners. For this POC, "Heavy Rotation" is defined using Spotify's **Top Artists API** (https://developer.spotify.com/documentation/web-api/reference/get-users-top-artists-and-tracks), which identifies a fan as a heavy rotation listener if the target artist appears in their Top Artists list. Spotify calculates this based on affinity derived from total plays, recency, and listening patterns over configurable time ranges (4 weeks, 6 months, or 1 year).

#### ❌ Out of Scope:

* [cite_start]**Full-Scale Architecture:** We will **not** be building the production AWS pipeline (e.g., AWS Lambda, SQS, Step Functions) during this phase[cite: 1].
* [cite_start]**Full Dataset:** The POC will **not** process the entire 73M+ fan dataset[cite: 1].
* [cite_start]**Production-Ready Features:** This POC will not include a CI/CD pipeline, comprehensive monitoring and alerting, or an automated token refresh mechanism[cite: 1]. These will be addressed in the full project build.

---

## 4. 🏗️ POC Architecture & Workflow

[cite_start]The POC leverages a serverless-first approach within Snowflake, decoupling the interactive UI from the heavy data processing[cite: 2454].

**User Workflow:**
1.  A user selects an artist in the Streamlit app and clicks "Run Analysis."
2.  The app triggers a Snowflake Task to start running in the background.
3.  The UI immediately becomes responsive, showing a "Processing..." status.
4.  The Snowflake Task orchestrates the process of fetching data for all 10,000 fans by calling a Python UDF that connects to the Spotify API.
5.  Once the data is collected and aggregated, the Streamlit app detects the job completion and automatically displays the results.

```mermaid
graph TD
    subgraph "Snowflake Environment"
        A[Streamlit App] -- 1. Triggers task --> B(Snowflake Task);
        A -- 5. Polls for status --> D[POC_JOB_STATUS Table];
        B -- 2. Executes --> C{Orchestration Stored Procedure};
        C -- 3. For each fan, calls --> E[Python UDF];
        C -- 4. Stores raw data --> F[POC_RAW_LISTENING_DATA Table];
        C -- 6. Stores aggregated results --> G[POC_AGG_HEAVY_LISTENERS Table];
        C -- Updates status --> D;
        E -- Fetches token --> H[POC_FAN_TOKENS Table];
    end
    
    subgraph "External"
      E -- Makes API call --> I((Spotify API));
    end

    style I fill:#1DB954,stroke:#fff,color:#fff