# PDP Backfill Upload Application

A full-stack application for uploading CSV and JSON files to AWS S3 buckets to trigger PDP (Permissions Platform) backfill operations.

## Project Structure

This is a monorepo containing:

- **`frontend/`** - React + TypeScript web application
- **`api/`** - FastAPI backend service
- **`_tasks/`** - Project documentation and planning

## Architecture

The application uses a backend proxy architecture where the frontend sends files to the FastAPI backend via HTTP, and the backend uploads them to S3 using boto3. This approach avoids CORS configuration requirements on S3 buckets and keeps AWS credentials server-side.

```
┌─────────────────────────────────────────────────────────────────┐
│                     Application Architecture                     │
└─────────────────────────────────────────────────────────────────┘

    User Browser
    ┌─────────────┐
    │             │
    │  Frontend   │
    │  (React)    │
    │             │
    │  :3000      │
    └──────┬──────┘
           │
           │ HTTP POST /upload/
           │ (multipart/form-data)
           │ - csv_files[]
           │ - manifest_file
           │ - folder_path
           │ - environment
           │
           ▼
    ┌─────────────┐
    │             │
    │   Backend   │
    │  (FastAPI)  │
    │             │
    │  :8888      │
    └──────┬──────┘
           │
           │ AWS SDK (boto3)
           │ - put_object()
           │ - Uses AWS credentials
           │   from environment
           │
           ▼
    ┌─────────────┐
    │             │
    │   AWS S3    │
    │   Buckets   │
    │             │
    │  qa/prod    │
    └─────────────┘

Upload Flow:
1. User selects files in browser (frontend)
2. Frontend validates files and folder path
3. Frontend sends files to backend via HTTP
4. Backend receives files and uploads to S3
5. Backend returns success/error response
6. Frontend displays upload status to user

Key Benefits:
- No CORS configuration needed on S3 buckets
- AWS credentials never exposed to browser
- Natural integration with awsume for credentials
- Clear separation of concerns
```

## Quick Start

The easiest way to run the full application is with Docker Compose:

```bash
# From project root
export GITHUB_NPM_TOKEN=<your-token>
awsume permissions-platform-qa-generic
docker-compose up --build
```

Access:
- Frontend: http://localhost:3000
- Backend API: http://localhost:8888
- API Docs: http://localhost:8888/docs

### Local Development

For local development without Docker, see:
- **Backend**: [api/README.md](./api/README.md)
- **Frontend**: [frontend/README.md](./frontend/README.md)

## AWS Credentials

The backend requires AWS credentials to upload to S3. Use `awsume` to assume the appropriate role:

```bash
# For QA environment
awsume permissions-platform-qa-generic

# For Production environment
awsume permissions-platform-prod-generic
```

The backend will automatically use credentials from the environment (set by awsume).

## Environment Variables

### Backend (api/.env)

```bash
AWS_REGION=us-east-1
QA_BUCKET=qa-pdp-backfill
PROD_BUCKET=prod-pdp-backfill
```

### Frontend (frontend/.env)

```bash
VITE_API_URL=http://localhost:8888
VITE_QA_BUCKET=qa-pdp-backfill
VITE_PROD_BUCKET=prod-pdp-backfill
```

## Development Workflow

### Backend Development

```bash
cd api

# Install dependencies
uv sync

# Run linting
make lint

# Run tests
make test_unit

# Start dev server
make dev
```

### Frontend Development

```bash
cd frontend

# Install dependencies
pnpm install

# Run all checks (lint + format + typecheck)
pnpm check

# Format code
pnpm format

# Start dev server
pnpm start
```

## Testing

### Manual Testing

1. Start backend: `cd api && make dev`
2. Start frontend: `cd frontend && pnpm start`
3. Navigate to http://localhost:3000
4. Select environment (QA or Production)
5. Choose CSV files and manifest.json
6. Enter folder path (e.g., "PP-1055")
7. Review files and upload
8. Verify files appear in S3 bucket

### Verify S3 Upload

```bash
awsume permissions-platform-qa-generic
aws s3 ls s3://qa-pdp-backfill/<folder-path>/
```

## Key Features

- **Multi-step workflow**: 6-step guided process for file upload
- **Validation**: Multiple validation layers (frontend + backend)
- **Environment safety**: Production uploads require explicit confirmation
- **Error handling**: Comprehensive error messages and recovery
- **Progress tracking**: Real-time upload progress feedback

## Technology Stack

### Frontend
- React 18.3.1 with TypeScript 5.9.0
- React Router DOM v5.3.4
- Vite build tool
- @theorchard/suite-components (internal UI library)
- Zod v4.0.0 for validation

### Backend
- FastAPI 0.110.0
- boto3 (AWS SDK for Python)
- Pydantic for schema validation
- uvicorn ASGI server

## Project Documentation

See `CLAUDE.md` for detailed architecture documentation and development guidelines.

See `_tasks/` directory for implementation plans and task tracking.

## Support

For issues or questions, refer to the internal Slack channel or Datadog dashboard (links in `.env` files).