# CLAUDE.md — ows-charts

This file provides guidance for AI assistants working in the ows-charts repository.

## Project Overview

**ows-charts** is a Python Flask REST API that serves as the Data Access Layer (DAL) to Snowflake for music chart data. It provides chart rankings, chart metadata, and New Music Friday (NMF) analytics across streaming platforms. Consumed by `graphql-analytics` and other internal services.

## Essential Commands

```bash
# Development
python dev.py               # Dev server with hot-reload

# Testing
make test_unit              # pytest unit tests with coverage
make test_integration       # Integration tests (requires Snowflake)
make test                   # All tests

# Linting
make lint                   # flake8 charts/ stubs/ tests/

# Type Checking
make mypy                   # mypy on select directories (progressive adoption)

# Full CI job
make unit_lint_job          # clean + pip_dev + lint + test_unit + mypy
make integration_job        # Integration tests against QA
```

To run a single test file:
```bash
pytest tests/unit/test_handlers.py -v
pytest tests/unit/test_handlers.py::test_specific_function -v
```

## Setup

Copy `.env.shadow` to `.env` before running locally: `cp .env.shadow .env`

Snowflake access requires private key in `~/.ssh/snowflake/` (local) or AWS Secrets Manager (QA/prod).

## Architecture

### Tech Stack

| Layer | Technology |
|-------|-----------|
| Framework | Flask 2.2.5 |
| Language | Python 3.11 |
| Database | Snowflake (via snowflake-connector-sqlalchemy) |
| ORM | Flask-SQLAlchemy 2.5 (abstraction over SQLAlchemy) |
| Validation | Marshmallow + Pydantic 2.11 |
| Caching | Redis 5.0 (with FakeRedis fallback) |
| Package Manager | pip (requirements.txt + requirements-dev.txt) |
| APM | Datadog (ddtrace) |
| Error Tracking | Sentry (raven) |
| Authorization | owsrequest |
| Feature Flags | pythonfeatures |
| API Docs | flasgger, flask-apispec, flask-pydantic-api |
| Type Checking | mypy (progressive adoption, with stub directory) |

### Directory Structure

```
charts/
├── api.py                       # Flask + SQLAlchemy setup
├── config.py                    # Configuration
├── handlers.py                  # Main request handlers (~22KB)
├── new_music_friday_handlers.py # NMF-specific endpoints
├── features.py                  # Feature flag logic
├── connectors/                  # Service connectors
│   ├── snowflake.py             # Snowflake connection (PKI auth)
│   └── redis.py                 # Redis cache (FakeRedis fallback)
├── constants/                   # ~13 constant modules
│   ├── dates.py                 # Date calculations
│   ├── new_music_friday.py      # NMF-specific constants
│   └── stores.py                # Store IDs
├── models/                      # SQLAlchemy ORM models
│   ├── chart.py
│   └── chart_ranking.py
├── schemas/                     # Marshmallow schemas (~13 modules)
├── logic/                       # Business logic (~17 modules)
│   ├── cache_prime.py           # Cache warming on deployment
│   ├── new_music_friday.py      # NMF logic
│   ├── dataloaders/             # DataLoader pattern
│   └── permissions.py           # Permission logic
├── services/                    # Service layer (~6 modules)
│   └── cache_service.py
└── utils/                       # Utilities (DB, SQL formatting, dataloader)

tests/
├── unit/                        # Unit tests
│   ├── conftest.py
│   ├── fixtures/
│   └── logic/
└── integration/                 # Integration tests

stubs/                           # mypy type stubs
spec/                            # OpenAPI specifications
```

### Data Flow

```
HTTP Request → owsrequest (auth) → Handler → Logic → SQLAlchemy Model → Snowflake
                                        │       ↑
                                        │   Redis cache
                                        ↓
                                  Marshmallow/Pydantic → JSON Response
```

## Key Patterns

### ORM-Based Queries

Unlike `ows-playlist` (JinjaSQL templates) and `ows-analytics` (direct connector), this service uses **Flask-SQLAlchemy ORM models** for Snowflake queries. Models are defined in `models/` using SQLAlchemy's declarative base.

### Dual Validation

Uses both **Marshmallow** (legacy, response serialization) and **Pydantic 2** (modern, request validation via flask-pydantic-api). New endpoints should prefer Pydantic.

### New Music Friday (NMF)

Dedicated handlers and logic for platform-specific new release analytics (Spotify NMF, Apple Music, etc.). Separate date calculation logic in `constants/dates.py` and `logic/new_music_friday_date.py`.

### Cache Priming

`logic/cache_prime.py` pre-populates Redis cache after deployments to avoid cold-start latency for common queries.

### Progressive Type Checking

mypy is being adopted incrementally. Type stubs in `stubs/` directory provide type information for untyped dependencies. The Makefile runs mypy on select directories only.

### SQLite for Tests

Unit tests use SQLite in-memory database instead of Snowflake. Configured in `config.py` test environment.

## Environment Variables

| Variable | Purpose |
|----------|---------|
| `ENV` | `qa` / `prod` |
| `SNOWFLAKE_ACCOUNT` | Snowflake account |
| `SNOWFLAKE_USER` | Snowflake username |
| `SNOWFLAKE_ROLE` | Snowflake role |
| `SNOWFLAKE_WAREHOUSE` | Snowflake warehouse |
| `SNOWFLAKE_DATABASE` | Snowflake database |
| `SNOWFLAKE_SCHEMA` | Snowflake schema |
| `REDIS_HOST` | Redis host |
| `SPLIT_API_KEY` | Split.io feature flags |
| `SENTRY_DSN` | Sentry error tracking |

## Testing

- **Unit tests**: Mock Snowflake via SQLite in-memory, mock Redis via FakeRedis
- **Fixtures**: Located in `tests/unit/fixtures/`
- **Coverage target**: 80%+ (enforced in CI)
- **Integration tests**: Run against real Snowflake

## Docker

Multi-stage: `base` (python:3.11-slim-bullseye) → `deploy` | `pr_tests`.

Docker Compose includes local Redis for development.