# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

PDP Backfill Upload Application - A full-stack monorepo application for uploading CSV and JSON files to AWS S3 buckets to trigger PDP (Permissions Platform) backfill operations. The application consists of a React frontend and FastAPI backend, providing a multi-step workflow for file selection, validation, and upload to QA or Production environments.

**Architecture**: Backend proxy pattern where the frontend sends files to FastAPI backend via HTTP, and the backend uploads to S3 using boto3. This avoids CORS configuration on S3 buckets and keeps AWS credentials server-side.

## Development Commands

### Backend (FastAPI)

```bash
cd api

# Install dependencies
uv sync

# Local development (requires awsume credentials)
awsume permissions-platform-qa-generic
make dev  # Runs on http://localhost:8888

# Code quality
make lint           # Run ruff linter
make test_unit      # Run unit tests

# View API docs
# Navigate to http://localhost:8888/docs
```

### Frontend (React)

```bash
cd frontend

# Install dependencies (requires GITHUB_NPM_TOKEN environment variable)
pnpm install

# Local development (backend must be running)
pnpm start  # Opens http://localhost:3000

# Code quality
pnpm check          # Run all checks (lint + format check + typecheck)
pnpm lint           # Run Biomejs linter
pnpm lint:fix       # Auto-fix linting issues
pnpm format         # Format code with Biomejs
pnpm format:check   # Check code formatting
pnpm typecheck      # Run TypeScript type checking

# Build
pnpm build  # Build for production (output in /build directory)
```

### Docker Development (Both Services)

```bash
# From project root
export GITHUB_NPM_TOKEN=<token>
docker-compose up --build

# Access:
# - Frontend: http://localhost:3000
# - Backend: http://localhost:8888
# - API Docs: http://localhost:8888/docs
```

### Important Notes
- **Backend**: Python 3.11+, uses uv for dependency management
- **Frontend**: Node.js >= 18.0.0, uses pnpm for package management
- No automated frontend tests currently exist
- Backend has unit tests (run with `make test_unit`)
- Frontend uses @theorchard/frontend-cli with Vite plugin for build tooling

## Architecture Overview

### Technology Stack

**Frontend:**
- React 18.3.1 with TypeScript 5.9.0
- React Router DOM v5.3.4
- Build Tool: Vite via @theorchard/frontend-cli-vite
- Design System: @theorchard/suite-components (internal UI library)
- Validation: Zod v4.0.0 for runtime schema validation
- Linting/Formatting: Biomejs v2.2.4

**Backend:**
- FastAPI 0.110.0
- boto3 (AWS SDK for Python) for S3 uploads
- Pydantic for schema validation
- uvicorn ASGI server

### Project Structure (Monorepo)

```
pdp-backfill-app/
├── frontend/                        # React application
│   ├── src/
│   │   ├── components/BackfillUploader/  # Core multi-step upload workflow
│   │   │   ├── BackfillUploader.tsx      # Main orchestrator (state machine)
│   │   │   ├── FileSelector.tsx          # CSV and manifest file selection
│   │   │   ├── FolderPathInput.tsx       # S3 folder path input with validation
│   │   │   ├── FileReview.tsx            # Pre-upload review and validation
│   │   │   └── UploadStatus.tsx          # Progress and completion feedback
│   │   ├── pages/home/                   # Landing page with instructions
│   │   ├── services/                     # Backend API communication
│   │   │   ├── apiConfig.ts              # Backend API configuration
│   │   │   └── uploadService.ts          # Upload operations via backend API
│   │   ├── types/                        # TypeScript type definitions
│   │   │   ├── backfill.ts               # Backfill domain types
│   │   │   └── manifest.ts               # Manifest file structure types
│   │   ├── schemas/                      # Zod validation schemas
│   │   │   └── manifestSchema.ts         # Manifest JSON validation
│   │   └── styles/                       # Global CSS
│   ├── package.json
│   ├── tsconfig.json
│   └── biome.json
├── api/                             # FastAPI backend
│   ├── backfillapi/
│   │   ├── api/
│   │   │   ├── routers/
│   │   │   │   ├── upload.py             # Upload endpoint
│   │   │   │   └── infra.py              # Health checks
│   │   │   ├── schemas/
│   │   │   │   └── upload.py             # Pydantic schemas
│   │   │   └── main.py                   # FastAPI app
│   │   ├── services/
│   │   │   └── s3_service.py             # S3 upload operations
│   │   └── config.py                     # Configuration
│   ├── pyproject.toml
│   └── Makefile
├── docker-compose.yml               # Orchestrates both services
├── README.md                        # Project overview and quick start
├── CLAUDE.md                        # Detailed architecture and guidelines
└── _tasks/                          # Project documentation and planning
```

### Key Application Flow

The application implements a 6-step state machine workflow:

**Step 1: select-environment** → User chooses QA or Production
**Step 2: select-files** → Select CSV files (permission data) and one manifest.json
**Step 3: enter-path** → Specify S3 folder path (e.g., "PP-1055")
**Step 4: review** → Validate manifest structure and file references
**Step 5: upload** → Upload CSV files first, then manifest (triggers backfill)
**Step 6: complete** → Display success/error status

### State Management Pattern
- Uses local React state with useState hooks (no Redux/Context needed)
- BackfillUploader component maintains all workflow state
- Unidirectional data flow via props and callbacks
- State includes: environment, selectedFiles, folderPath, uploadProgress, uploadResult

### Validation Pipeline

Multiple validation layers prevent invalid uploads:

1. **FileSelector**: Immediate validation on file selection
   - CSV extension check
   - JSON parseability check

2. **FolderPathInput**: Real-time path validation
   - No leading/trailing slashes
   - No reserved paths (root, hidden folders)
   - No invalid S3 characters

3. **FileReview**: Pre-upload validation
   - Manifest structure against Zod schema
   - Manifest bucket matches selected environment
   - Manifest keys reference uploaded CSV files

4. **s3Service**: Final validation before AWS operations
   - AWS credentials validation
   - Folder path validation
   - Custom error handling with S3UploadError class

### AWS Integration Architecture

**Backend Proxy Pattern**:
- Frontend sends files to FastAPI backend via HTTP POST
- Backend uploads files to S3 using boto3 (AWS SDK for Python)
- AWS credentials stay server-side (read from environment via awsume)
- No CORS configuration needed on S3 buckets
- No AWS credentials exposed to browser

**Bucket Configuration**:
- QA: `qa-pdp-backfill`
- Production: `prod-pdp-backfill`
- Region: `us-east-1` (default)

**Upload Flow**:
1. User selects files in frontend (React)
2. Frontend validates files and folder path
3. Frontend sends multipart/form-data to backend `/upload/` endpoint
4. Backend validates request parameters
5. Backend uploads CSV files to S3 sequentially
6. Backend uploads manifest file last (triggers backfill process)
7. Backend returns success/error response
8. Frontend displays upload status to user

### Error Handling Architecture

**Custom Error Class**: S3UploadError with structured error information
- Distinguishes credential, permission, bucket, and upload errors
- Provides user-friendly error messages
- Error recovery via Reset button

**Error Display**: UploadStatus component shows detailed error messages with context

## Code Quality Standards

### Biomejs Configuration
- 4-space indentation, 120-char line width
- Single quotes, always semicolons, trailing commas
- **No default exports** (except lazy-loaded components)
- Import type checking required
- Git-aware (respects .gitignore)

### TypeScript
- Strict mode enabled
- Path aliases: `src/*` → `./src/*`
- ES2020 target
- Module resolution: bundler

## Environment Variables

### Backend (api/.env)

```bash
AWS_REGION=us-east-1                              # AWS region
QA_BUCKET=qa-pdp-backfill                        # QA S3 bucket
PROD_BUCKET=prod-pdp-backfill                    # Production S3 bucket
```

Optional monitoring/support links:
```bash
DATADOG_DASHBOARD_URL=...                        # Monitoring dashboard
SLACK_CHANNEL_URL=...                            # Support channel
```

### Frontend (frontend/.env)

```bash
VITE_API_URL=http://localhost:8888               # Backend API URL
VITE_QA_BUCKET=qa-pdp-backfill                  # QA bucket (for display only)
VITE_PROD_BUCKET=prod-pdp-backfill              # Prod bucket (for display only)
```

Optional monitoring/support links:
```bash
VITE_DATADOG_DASHBOARD_URL=...                   # Monitoring dashboard
VITE_SLACK_CHANNEL_URL=...                       # Support channel
```

## AWS Credentials Setup

The backend requires AWS credentials to upload to S3. Credentials are read from the environment using the standard AWS credential chain.

### Local Development Workflow (Recommended)

1. **Install AWS CLI and awsume**: https://aws.amazon.com/cli/
   ```bash
   pip install awsume
   ```

2. **Assume AWS role with awsume**:
   ```bash
   # For QA environment
   awsume permissions-platform-qa-generic

   # For Production environment
   awsume permissions-platform-prod-generic
   ```

3. **Start the backend**:
   ```bash
   cd api
   make dev  # Runs on http://localhost:8888
   ```

4. **Start the frontend** (in another terminal):
   ```bash
   cd frontend
   pnpm start  # Runs on http://localhost:3000
   ```

5. **Access the app**: http://localhost:3000

### How Credentials Work

- Backend runs server-side and reads AWS credentials from environment variables
- awsume sets `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN`
- boto3 automatically picks up these credentials
- Frontend never sees or handles AWS credentials
- Frontend only sends files to backend via HTTP

### Troubleshooting

**Error: "AWS credentials not found" or "Access denied"**

This means the backend cannot access AWS credentials. To fix:

1. Verify awsume credentials are set:
   ```bash
   echo $AWS_ACCESS_KEY_ID
   ```
   If empty, run `awsume permissions-platform-qa-generic` again.

2. Restart the backend:
   ```bash
   cd api
   # Kill any running dev servers
   pkill -f "uvicorn"

   # Start fresh (awsume credentials will be inherited)
   make dev
   ```

3. Verify credentials in backend logs:
   - Backend will log credential validation errors
   - Check terminal output for detailed error messages

**Credentials expired during use:**
- Awsume credentials typically expire after 1-12 hours
- Run `awsume` again in the terminal where you'll start the backend
- Restart the backend to pick up new credentials

### Docker Development

When using Docker Compose:
- The docker-compose.yml mounts `~/.aws` directory into backend container
- Run `awsume` before `docker-compose up` to set credentials
- Backend container inherits credentials via environment variables

## Working with Services

### Frontend Services

#### apiConfig.ts
Backend API configuration:
- `getApiUrl()` - Returns backend API base URL
- `getUploadEndpoint()` - Returns upload endpoint URL
- Reads from `VITE_API_URL` environment variable

#### uploadService.ts
Upload operations via backend API:
- `uploadFiles()` - Sends files to backend via HTTP POST
- `validateFolderPath()` - Client-side folder path validation
- Creates multipart/form-data payload with CSV files, manifest, folder path, and environment
- Handles HTTP errors and returns structured UploadResult

### Backend Services

#### api/backfillapi/services/s3_service.py
S3 upload operations with comprehensive error handling:
- `upload_files()` - Main orchestrator (CSV first, then manifest)
- `upload_file_to_s3()` - Single file upload with boto3
- `validate_folder_path()` - Server-side folder path validation
- `construct_s3_key()` - Builds full S3 path from folder + filename
- `S3UploadError` - Custom exception class for structured error handling

#### api/backfillapi/api/routers/upload.py
Upload API endpoint:
- `POST /upload/` - Receives multipart/form-data from frontend
- Validates environment and folder path
- Reads file contents from upload
- Calls s3_service.upload_files()
- Returns UploadResponse with status and uploaded file keys

#### api/backfillapi/config.py
Backend configuration:
- `AWS_REGION` - AWS region (default: us-east-1)
- `QA_BUCKET` - QA S3 bucket name
- `PROD_BUCKET` - Production S3 bucket name

## Component Architecture

### BackfillUploader (Main Orchestrator)
Manages workflow state machine with 6 steps. Coordinates all child components and handles state transitions.

### FileSelector
File input handling with validation feedback. Supports multiple CSV files and one manifest.json.

### FolderPathInput
S3 folder path input with real-time validation and preview of full S3 paths.

### FileReview
Pre-upload validation display with detailed error messages. Validates manifest structure and cross-references with selected CSV files. Shows production environment warnings.

### UploadStatus
Progress tracking and completion feedback. Displays file-by-file upload progress and final success/error status.

## Important Constraints and Patterns

### Manifest Upload Strategy
**Critical**: The manifest file must be uploaded last because it triggers the backfill process. CSV files are uploaded first, then the manifest references them via S3 keys.

### Environment Safety
Production uploads require explicit confirmation. The UI displays prominent warnings when Production is selected.

### Accessibility
Components use semantic HTML, ARIA labels, live regions, and keyboard navigation support throughout.

### No Testing Infrastructure
This codebase does not currently have automated tests. Changes should be manually tested via the UI workflow.

## Modification Guidelines

When modifying components:
- Maintain the 6-step workflow sequence
- Preserve validation at all layers (client, service, schema)
- Keep unidirectional data flow (props down, callbacks up)
- Follow Biomejs rules (no default exports, import type checking)
- Use TypeScript strict mode (all types must be explicit)
- Test manually through complete workflow in both QA and Production modes

When modifying services:
- Maintain CSV-first, manifest-last upload order
- Preserve S3UploadError structure for consistent error handling
- Keep validation functions pure (no side effects)
- Document AWS SDK interactions with JSDoc comments

When adding new validations:
- Add Zod schema updates in `schemas/manifestSchema.ts`
- Add runtime checks in `s3Service.ts`
- Update UI validation feedback in relevant component
- Update TypeScript types in `types/` directory