# songwhip-lookup This service provides a unified interface for looking up music entities (artists, albums, tracks) across various streaming platforms. ## Supported Services - Amazon - [Apple Music](lib/services/applemusic/README.md) - Audiomack - Audius - AWA - Bandcamp - Bandsintown - Deezer - Discogs - Gaana - JioSaavn - LINE Music - MusicBrainz - Napster - Orchard - [Pandora](lib/services/pandora/README.md) - Qobuz - SoundCloud - Spotify - Ticketmaster - Tidal - Yandex Music - [YouTube](lib/services/youtube/README.md) - YouTube Music ## How It Works The service is hosted in Vercel as serverless functions, exposing API endpoints for looking up and searching music entities. The API has three main endpoints: - `/lookup?url={url}`: Fetch an entity by a service store URL - `/upc?upc={upc}`: Find albums using an UPC (Universal Product Code) - `/isrc?isrc={isrc}`: Find tracks using an ISRC (International Standard Recording Code) ### The "Source Item" When looking up or searching for entities, the service always begins by finding one or more "Source Items". The source item is the initial entity retrieved from the source service (e.g., Spotify, Apple Music) based on the provided URL, UPC, or ISRC. This source item is then used to find equivalent entities across other supported services. For URL lookups, there is a single source item corresponding to the parsed URL. For UPC and ISRC searches, there may be multiple source items returned from the initial search. ### Source Country When performing lookups or searches, the service uses a "source country" to tailor requests to the source service's API. This ensures that region-specific content and availability are respected. The source country is either specified in the request params, derived from the request headers, or defaults to "US" if not provided. **Note that there are only a few services that support country-specific requests. Most services will return global results regardless of the source country specified.** ### Architecture The service follows a **layered architecture** with clear separation of concerns: ``` ┌─────────────────────────────────────────────────────┐ │ API Endpoints │ │ (Vercel Serverless Functions) │ └─────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────┐ │ Business Logic Layer │ │ • Lookup orchestration (by URL, UPC, ISRC) │ │ • Cross-service search coordination │ └─────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────┐ │ Service Integrations │ │ Each service follows a 4-layer pattern: │ │ ┌───────────────────────────────────────────────┐ │ │ │ 1. API Layer - Raw HTTP & authentication │ │ │ │ 2. Mapper Layer - Data transformation │ │ │ │ 3. Lookup Layer - Entity lookup logic │ │ │ │ 4. Search Layer - Query-based discovery │ │ │ └───────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────┘ ↓ ┌─────────────────────────────────────────────────────┐ │ Shared Utilities │ │ • Entity models (Artist, Album, Track) │ │ • Cached HTTP client │ │ • Reducers │ │ • String matching & cleaning │ │ • Error reporting │ └─────────────────────────────────────────────────────┘ ``` ### Request Pipeline Here's how a typical lookup or search request flows through the system: ``` ┌──────────────────────────────────────────────────────────────────┐ │ 1. API ENDPOINT (Vercel Serverless Function) │ │ • Parse request (URL, UPC, ISRC, or search query) │ │ • Determine source country │ └──────────────────────────────────────────────────────────────────┘ ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 2. FETCH SOURCE ITEM(S) │ │ • Identify source service from URL/query │ │ • Parse entity type and ID │ │ • Call source service's lookup/search │ │ ┌────────────────────────────────────────────────────────┐ │ │ │ Source Service (e.g., Spotify) │ │ │ │ → API Layer: Make HTTP request │ │ │ │ → Mapper Layer: Transform to Songwhip entity │ │ │ │ → Return: Artist/Album/Track with source data │ │ │ └────────────────────────────────────────────────────────┘ │ │ • Result: Source Item(s) with core metadata │ └──────────────────────────────────────────────────────────────────┘ ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 3. SEARCH ALL SERVICES │ │ • For each supported service (in parallel): │ │ ┌────────────────────────────────────────────────────────┐ │ │ │ Service A (e.g., Apple Music) │ │ │ │ → Search Layer: Query by entity name/metadata │ │ │ │ → API Layer: Make search request │ │ │ │ → Mapper Layer: Transform results to entities │ │ │ │ → Return: Candidate entities │ │ │ └────────────────────────────────────────────────────────┘ │ │ ┌────────────────────────────────────────────────────────┐ │ │ │ Service B (e.g., Pandora) │ │ │ │ → Search Layer: Query by entity name/metadata │ │ │ │ → API Layer: Make search request │ │ │ │ → Mapper Layer: Transform results to entities │ │ │ │ → Return: Candidate entities │ │ │ └────────────────────────────────────────────────────────┘ │ │ │ ... (all other services) │ │ • Result: Array of candidate entities per service │ └──────────────────────────────────────────────────────────────────┘ ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 4. MATCH & SCORE │ │ • For each service's results: │ │ - Compare unique identifiers with source (UPC/ISRC) │ │ - Compare candidate names with source item name │ │ - Calculate string similarity scores │ │ - Check metadata alignment (artists, album, duration) │ │ - Apply service-specific matching rules │ │ • Select best match per service │ │ • Result: One matched entity per service │ └──────────────────────────────────────────────────────────────────┘ ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 5. REDUCE TO UNIFIED ENTITY │ │ • Merge all matched entities into one: │ │ - Combine links from all services │ │ - Select best metadata (name, image, release date) │ │ - Aggregate nested entities (artists, albums, tracks) │ │ - Deduplicate and normalize data │ │ • Result: Single unified entity with all service links │ └──────────────────────────────────────────────────────────────────┘ ↓ ┌──────────────────────────────────────────────────────────────────┐ │ 6. SERIALIZE & RESPOND │ │ • Convert entity to API response format │ │ • Include metadata (source service, country, timestamps) │ │ • Return JSON response to client │ │ ┌────────────────────────────────────────────────────────┐ │ │ │ Response Example: │ │ │ │ { │ │ │ │ "type": "artist", │ │ │ │ "name": "Artist Name", │ │ │ │ "image": "https://...", │ │ │ │ "links": { │ │ │ │ "spotify": ["https://open.spotify.com/..."], │ │ │ │ "appleMusic": ["https://music.apple.com/..."], │ │ │ │ "pandora": ["https://www.pandora.com/..."], │ │ │ │ ... │ │ │ │ } │ │ │ │ } │ │ │ └────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────┘ ``` **Key Points**: - **Parallel Processing**: Step 3 invokes all services concurrently for speed - **Fault Tolerance**: Failed service lookups don't block the entire pipeline - **Caching**: HTTP responses are cached to reduce API calls - **Matching Logic**: String matching and metadata comparison ensures accurate results - **Reducer Pattern**: Multiple entities are merged into a single unified entity ### Error Handling Errors are logged to Sentry through the [handler](lib/handler.ts) in the API layer, or by directly using the reporter utility. Individual service errors do not block the overall request; best-effort results are returned. Service level errors are sent to Sentry for monitoring. **Note that no sensitive information (API keys, tokens) should ever be logged.** ### Principles - **Layered Design**: Clear separation between API integration, data transformation, and business logic - **Portable Components**: Each layer can be tested and modified independently - **Type Safety**: Comprehensive TypeScript types throughout - **Caching**: Built-in HTTP caching to minimize external API calls - **Testability**: Mocked API responses for deterministic tests See [Service Implementation Pattern](.github/copilot-instructions.md#service-implementation-pattern) for detailed architecture guidelines. ## Getting Started ### Prerequisites - Node.js configured repo\ https://www.notion.so/Node-js-project-setup-guide-86feadffeb304962b06e6b1bb8f09213 ### Installation ```bash # Install dependencies yarn install # Create a .env file in the root directory cp .env.shadow .env ``` ### Running Locally Start the development server: ```bash yarn dev ``` This will start a local express server mapping the routes to the serverless functions in the `api/` directory. The service will be available at `http://localhost:5001` (or the port specified by the `PORT` env var). **To be able to run tests involving real API calls, you need to add API keys or tokens to the `.env` file.** The keys are found in the songwhip-lookup Vercel project settings. If you do not have access, please reach out to a team member. **Debugging using VSCode**: Add a launch configuration to start the server with the debugger attached ``` // .vscode/launch.json { "configurations": [ { "name": "Debug songwhip-lookup", "request": "launch", "runtimeArgs": ["dev"], "runtimeExecutable": "yarn", "skipFiles": ["/**"], "type": "node" } ] } ``` ## Testing ### Run All Tests ```bash # Run linting, type checking, and all unit tests yarn test ``` ### Run Specific Test Suites ```bash # Run only unit tests yarn test:jest # Run only linting yarn test:lint # Run only type checking yarn test:types # Run unit tests for a specific service yarn test:jest pandora # Run unit tests in watch mode yarn test:jest pandora --watch ``` ## Updating Service Test Payloads Test payloads are mock API responses stored in `test/unit/tests/services//__payloads__/` directories. These should be updated when: - A service's API response format changes - New fields are added to API responses - Test coverage is expanded to new scenarios ### Automated Update Scripts Services should have scripts to automatically fetch and update test payloads: ```bash # General pattern yarn # Examples yarn pandora # Update Pandora test payloads ``` ### Payload Guidelines - **Use realistic data**: Payloads should closely resemble actual API responses - **Cover edge cases**: Include examples with missing fields, empty arrays, special characters - **Keep them current**: Update payloads when the external API changes - **Descriptive names**: Use clear filenames like `artist-with-no-albums.json`, `track-explicit.json` - **Organize by type**: Separate `lookup/` and `search/` payloads into subdirectories ## Code Style and Conventions This project follows strict TypeScript conventions: - **Named exports only** (except service `index.ts` files) - **Explicit return types** on all functions - **Type imports** using `import type` syntax - **camelCase** for files (PascalCase only for classes) - **No `any` types** - always use specific types - **No `@ts-ignore`** - use `@ts-expect-error` only when absolutely necessary See [Copilot Instructions](.github/copilot-instructions.md) for complete coding guidelines. ## Adding or Modifying Services When adding or modifying service integrations: 1. Follow the [Service Implementation Pattern](.github/copilot-instructions.md#service-implementation-pattern) 2. Add comprehensive tests for all layers (API, mapper, lookup, search) 3. Update test payloads to reflect current API responses 4. Document the service in its `README.md` 5. Add the service to the "Supported Services" section above ## Resources - [Service Implementation Pattern](.github/copilot-instructions.md#service-implementation-pattern) - [Testing Guidelines](.github/copilot-instructions.md#testing-guidelines) - [Code Style Conventions](.github/copilot-instructions.md#code-style-conventions)