# Create Backend Upload Service - Implementation Plan

## Overview

This document outlines the plan to migrate from direct browser-to-S3 uploads (Option 1) to a FastAPI backend proxy
architecture (Option 2). The migration involves restructuring the project into a monorepo, rolling back VITE credential
changes, and implementing a backend upload service.

## Why We're Doing This

**Primary Reason**: Avoid CORS configuration requirements on QA and Production S3 buckets

**Additional Benefits**:

- Better security (credentials never exposed to browser)
- Natural awsume integration (backend runs server-side)
- Cleaner architecture with clear frontend/backend separation
- Future extensibility for validation, logging, rate limiting

## Project Structure Change

### Current Structure (Flat)

```
pdp-backfill-app/
├── src/                 # Frontend source
├── api/                 # Backend (FastAPI)
├── package.json         # Frontend deps
├── tsconfig.json        # Frontend config
├── node_modules/        # Frontend deps
└── build/               # Frontend build output
```

### Target Structure (Monorepo)

```
pdp-backfill-app/
├── frontend/            # React application
│   ├── src/
│   ├── package.json
│   ├── tsconfig.json
│   ├── node_modules/
│   ├── build/
│   └── ... (all frontend files)
├── api/                 # FastAPI backend
│   ├── backfillapi/
│   ├── pyproject.toml
│   └── ... (all backend files)
├── _tasks/              # Documentation (stays at root)
├── _tmp/                # Temporary files (stays at root)
├── docker-compose.yml   # Orchestrates both services
├── README.md            # Updated with monorepo info
└── CLAUDE.md            # Updated with new architecture
```

## Implementation Phases

---

## Phase 1: Restructure Project into Monorepo

### Goal

Move all frontend files from root into `frontend/` subdirectory while preserving git history.

### Steps

## Phase 1: Directory cleanup ✅

#### 1.1: Create `frontend/` Directory ✅

```bash
mkdir frontend
```

#### 1.2: Move Files Using Git (Preserves History) ✅

**Source code and configuration:**

```bash
git mv src frontend/src
git mv package.json frontend/package.json
git mv pnpm-lock.yaml frontend/pnpm-lock.yaml
git mv tsconfig.json frontend/tsconfig.json
git mv frontend.json frontend/frontend.json
git mv biome.json frontend/biome.json
git mv index.html frontend/index.html
```

**Docker files:**

```bash
git mv Dockerfile frontend/Dockerfile
git mv docker-compose.yml frontend/docker-compose.yml
```

**Config files:**

```bash
git mv .npmrc frontend/.npmrc
git mv .dockerignore frontend/.dockerignore
git mv .env frontend/.env
git mv .env.example frontend/.env.example
```

**Build artifacts (move, don't commit):**

```bash
# These should already be gitignored
mv node_modules frontend/node_modules 2>/dev/null || true
mv build frontend/build 2>/dev/null || true
mv .cache frontend/.cache 2>/dev/null || true
```

#### 1.3: Update `.gitignore` ✅

Update paths for frontend-specific ignores:

```gitignore
# Frontend specific
frontend/node_modules
frontend/build
frontend/.cache
frontend/dist
frontend/.env
frontend/.env.local

# API specific
api/.venv
api/__pycache__
api/.pytest_cache

# Keep existing root ignores
_tmp/
.DS_Store
.idea/
```

#### 1.4: Verify Structure ✅

```bash
ls -la frontend/
# Should see: src/, package.json, tsconfig.json, etc.

ls -la api/
# Should see: backfillapi/, pyproject.toml, etc.

ls -la .
# Should see: frontend/, api/, _tasks/, README.md, CLAUDE.md, etc.
```

---

## Phase 2: Rollback Option 1 (VITE Credentials) Changes ✅

### Goal

Remove all VITE credential-related code and AWS SDK dependencies from frontend.

### Files to Modify

#### 2.1: `frontend/src/services/awsConfig.ts` ✅

**Current state (Option 1 implementation):**

- Lines 31-63: Browser-specific credential logic with VITE_ env vars
- createS3Client() function that reads from import.meta.env

**Changes needed:**
This file can be deleted entirely since frontend no longer needs AWS SDK. Alternatively, create `apiConfig.ts` to
replace it.

**New file:** `frontend/src/services/apiConfig.ts`

```typescript
/**
 * Backend API configuration
 */
const API_BASE_URL = import.meta.env.VITE_API_URL || 'http://localhost:8888';

export function getApiUrl(): string {
    return API_BASE_URL;
}

export function getUploadEndpoint(): string {
    return `${API_BASE_URL}/upload/`;
}
```

#### 2.2: `frontend/package.json` ✅

**Line 8 - Rollback start script:**

**Current:**

```json
"start": "bash -c 'export VITE_AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID VITE_AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY VITE_AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN && npx @theorchard/frontend-cli dev'"
```

**Rollback to:**

```json
"start": "npx @theorchard/frontend-cli dev"
```

**Remove AWS SDK dependencies:**

```json
"dependencies": {
-    "@aws-sdk/client-s3": "^3.700.0",
-    "@aws-sdk/credential-providers": "^3.700.0",
"@theorchard/suite-components": "12.30.1",
...
}
```

Run after editing:

```bash
cd frontend
pnpm remove @aws-sdk/client-s3 @aws-sdk/credential-providers
```

#### 2.3: `frontend/.env.example` ✅

**Remove:**

```bash
# AWS Credentials (Option 1 - no longer used)
VITE_AWS_ACCESS_KEY_ID=
VITE_AWS_SECRET_ACCESS_KEY=
VITE_AWS_SESSION_TOKEN=
```

**Add:**

```bash
# Backend API URL
VITE_API_URL=http://localhost:8888

# S3 Bucket names (for display/validation only)
VITE_QA_BUCKET=qa-pdp-backfill
VITE_PROD_BUCKET=prod-pdp-backfill
```

#### 2.4: Delete Files No Longer Needed ✅

```bash
rm frontend/src/services/awsConfig.ts
rm cors-config.json
rm cors-config-prod.json
```

---

## Phase 3: Implement FastAPI Backend Upload Service ✅

### Goal

Create backend endpoints to handle file uploads to S3.

### Backend Dependencies

#### 3.1: Update `api/pyproject.toml` ✅

Add to `dependencies` array:

```toml
dependencies = [
    "audience-common~=3.3.1",
    "ddtrace~=2.7.4",
    "fastapi~=0.110.0",
    "httpx~=0.27.0",
    "jwtauth~=0.2.0",
    "sentry-sdk[fastapi]~=1.43.0",
    "uvicorn~=0.29.0",
    "boto3~=1.34.0",              # NEW: AWS SDK for Python
    "python-multipart~=0.0.9",     # NEW: For file upload handling
]
```

Install dependencies:

```bash
cd api
# Using uv (recommended)
uv sync

# Or using poetry (if available)
poetry install
```

### New Backend Files

#### 3.2: Create `api/backfillapi/services/__init__.py` ✅

```python
"""Services package."""
```

#### 3.3: Create `api/backfillapi/services/s3_service.py` ✅

Port logic from `frontend/src/services/s3Service.ts` to Python with boto3:

```python
"""S3 upload service - handles file uploads to S3 buckets."""

import logging
from typing import List, Optional, Callable
import boto3
from botocore.exceptions import ClientError, NoCredentialsError

from backfillapi import config

logger = logging.getLogger(__name__)


class S3UploadError(Exception):
    """Custom exception for S3 upload errors."""

    def __init__(self, message: str, code: Optional[str] = None, status_code: Optional[int] = None):
        super().__init__(message)
        self.code = code
        self.status_code = status_code


def get_s3_client():
    """Create boto3 S3 client with credentials from environment."""
    return boto3.client('s3', region_name=config.AWS_REGION)


def construct_s3_key(folder_path: str, filename: str) -> str:
    """
    Construct S3 key from folder path and filename.

    Args:
        folder_path: Folder path (e.g., "PP-1055")
        filename: File name

    Returns:
        Full S3 key (e.g., "PP-1055/permissions.csv")
    """
    clean_path = folder_path.strip().strip('/')
    return f"{clean_path}/{filename}" if clean_path else filename


def validate_folder_path(folder_path: str) -> Optional[str]:
    """
    Validate folder path for S3 upload.

    Returns:
        Error message if invalid, None if valid
    """
    trimmed = folder_path.strip()

    if not trimmed:
        return "Folder path cannot be empty"

    if trimmed.startswith('/') or trimmed.endswith('/'):
        return "Folder path should not start or end with a slash"

    # Reserved paths
    reserved_paths = ['pdp_qa_refresh', 'integration-test']
    if trimmed in reserved_paths:
        return f"'{trimmed}' is a reserved folder path and cannot be used"

    # Invalid S3 key characters
    invalid_chars = ['<', '>', '{', '}', '[', ']', '\\', '^', '%', '`', '|', '"']
    if any(char in trimmed for char in invalid_chars):
        return "Folder path contains invalid characters"

    return None


def upload_file_to_s3(
    s3_client,
    file_content: bytes,
    filename: str,
    s3_key: str,
    bucket: str,
    content_type: str = 'application/octet-stream'
) -> str:
    """
    Upload a single file to S3.

    Args:
        s3_client: Boto3 S3 client
        file_content: File content as bytes
        filename: Original filename (for error messages)
        s3_key: Full S3 key (path)
        bucket: S3 bucket name
        content_type: MIME type

    Returns:
        The S3 key of uploaded file

    Raises:
        S3UploadError: If upload fails
    """
    try:
        s3_client.put_object(
            Bucket=bucket,
            Key=s3_key,
            Body=file_content,
            ContentType=content_type
        )
        logger.info(f"Successfully uploaded {filename} to s3://{bucket}/{s3_key}")
        return s3_key

    except NoCredentialsError:
        raise S3UploadError(
            "AWS credentials not found. Please configure AWS CLI or check environment variables.",
            code="NoCredentials",
            status_code=401
        )
    except ClientError as e:
        error_code = e.response.get('Error', {}).get('Code', 'Unknown')

        if error_code == 'NoSuchBucket':
            raise S3UploadError(
                f"Bucket '{bucket}' does not exist",
                code="NoSuchBucket",
                status_code=404
            )
        elif error_code == 'AccessDenied':
            raise S3UploadError(
                f"Access denied when uploading '{filename}'. Check AWS permissions.",
                code="AccessDenied",
                status_code=403
            )
        else:
            raise S3UploadError(
                f"Failed to upload '{filename}': {str(e)}",
                code=error_code
            )


def upload_files(
    csv_files: List[tuple[str, bytes, str]],  # (filename, content, content_type)
    manifest_file: Optional[tuple[str, bytes, str]],
    folder_path: str,
    bucket: str,
    progress_callback: Optional[Callable[[str, int, int], None]] = None
) -> dict:
    """
    Upload CSV files and manifest to S3.

    Strategy: Upload CSV files first, then manifest last (manifest triggers backfill).

    Args:
        csv_files: List of (filename, content, content_type) tuples
        manifest_file: Optional (filename, content, content_type) tuple
        folder_path: S3 folder path
        bucket: S3 bucket name
        progress_callback: Optional callback(filename, current, total)

    Returns:
        {
            "status": "success" | "error",
            "message": str,
            "uploaded_files": List[str],
            "errors": Optional[List[str]]
        }
    """
    s3_client = get_s3_client()
    uploaded_files = []
    errors = []

    try:
        # Upload CSV files first
        total_csv = len(csv_files)
        for i, (filename, content, content_type) in enumerate(csv_files, start=1):
            s3_key = construct_s3_key(folder_path, filename)

            if progress_callback:
                progress_callback(filename, i, total_csv)

            upload_file_to_s3(s3_client, content, filename, s3_key, bucket, content_type)
            uploaded_files.append(s3_key)

        # Upload manifest last
        if manifest_file:
            filename, content, content_type = manifest_file
            s3_key = construct_s3_key(folder_path, filename)

            if progress_callback:
                progress_callback(filename, 1, 1)

            upload_file_to_s3(s3_client, content, filename, s3_key, bucket, content_type)
            uploaded_files.append(s3_key)

        return {
            "status": "success",
            "message": f"Successfully uploaded {len(uploaded_files)} file(s) to S3",
            "uploaded_files": uploaded_files
        }

    except S3UploadError as e:
        errors.append(str(e))
        return {
            "status": "error",
            "message": str(e),
            "uploaded_files": uploaded_files if uploaded_files else None,
            "errors": errors
        }
    except Exception as e:
        logger.exception("Unexpected error during upload")
        errors.append(str(e))
        return {
            "status": "error",
            "message": f"Upload failed: {str(e)}",
            "uploaded_files": uploaded_files if uploaded_files else None,
            "errors": errors
        }
```

#### 3.4: Create `api/backfillapi/api/schemas/upload.py` ✅

```python
"""Upload API schemas."""

from typing import List, Optional
from pydantic import BaseModel, Field


class UploadResponse(BaseModel):
    """Response from upload endpoint."""

    status: str = Field(..., description="Upload status: 'success' or 'error'")
    message: str = Field(..., description="Human-readable message")
    uploaded_files: Optional[List[str]] = Field(None, description="List of S3 keys for uploaded files")
    errors: Optional[List[str]] = Field(None, description="List of error messages if upload failed")


class ValidationErrorResponse(BaseModel):
    """Response for validation errors."""

    detail: str = Field(..., description="Error detail message")
```

#### 3.5: Create `api/backfillapi/api/routers/upload.py` ✅

```python
"""Upload router - handles file uploads to S3."""

import logging
from typing import List, Optional
from fastapi import APIRouter, UploadFile, File, Form, HTTPException, status

from backfillapi import config
from backfillapi.services import s3_service
from backfillapi.api.schemas import upload as schemas

logger = logging.getLogger(__name__)

router = APIRouter(tags=["upload"], prefix="/upload")


@router.post(
    "/",
    operation_id="upload_files",
    response_model=schemas.UploadResponse,
    status_code=status.HTTP_200_OK,
    summary="Upload CSV and manifest files to S3",
    description="Upload multiple CSV files and one manifest.json file to the specified S3 bucket and folder path."
)
async def upload_files(
    csv_files: List[UploadFile] = File(..., description="CSV files containing permission data"),
    manifest_file: UploadFile = File(..., description="Manifest JSON file"),
    folder_path: str = Form(..., description="S3 folder path (e.g., 'PP-1055')"),
    environment: str = Form(..., description="Target environment: 'qa' or 'prod'")
) -> schemas.UploadResponse:
    """
    Upload files to S3 bucket.

    CSV files are uploaded first, then the manifest file is uploaded last to trigger the backfill process.
    """
    # Validate environment
    if environment not in ['qa', 'prod']:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail=f"Invalid environment: {environment}. Must be 'qa' or 'prod'."
        )

    # Validate folder path
    folder_path_error = s3_service.validate_folder_path(folder_path)
    if folder_path_error:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail=folder_path_error
        )

    # Determine bucket
    bucket = config.QA_BUCKET if environment == 'qa' else config.PROD_BUCKET

    logger.info(f"Starting upload: {len(csv_files)} CSV files + manifest to {bucket}/{folder_path}")

    try:
        # Read CSV files
        csv_file_data = []
        for csv_file in csv_files:
            content = await csv_file.read()
            csv_file_data.append((csv_file.filename, content, csv_file.content_type or 'text/csv'))

        # Read manifest file
        manifest_content = await manifest_file.read()
        manifest_data = (manifest_file.filename, manifest_content, manifest_file.content_type or 'application/json')

        # Upload to S3
        result = s3_service.upload_files(
            csv_files=csv_file_data,
            manifest_file=manifest_data,
            folder_path=folder_path,
            bucket=bucket
        )

        if result['status'] == 'error':
            logger.error(f"Upload failed: {result['message']}")
            raise HTTPException(
                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
                detail=result['message']
            )

        logger.info(f"Upload successful: {len(result['uploaded_files'])} files")
        return schemas.UploadResponse(**result)

    except HTTPException:
        raise
    except Exception as e:
        logger.exception("Unexpected error during upload")
        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail=f"Upload failed: {str(e)}"
        )
```

#### 3.6: Update `api/backfillapi/config.py` ✅

Add S3 configuration constants:

```python
# Add after existing config

# S3 Configuration
AWS_REGION = os.environ.get("AWS_REGION", "us-east-1")
QA_BUCKET = os.environ.get("QA_BUCKET", "qa-pdp-backfill")
PROD_BUCKET = os.environ.get("PROD_BUCKET", "prod-pdp-backfill")
```

#### 3.7: Update `api/backfillapi/api/main.py` ✅

Import and include upload router:

```python
# Add to imports
from backfillapi.api.routers import infra, upload

# Add after app.include_router(infra.router)
app.include_router(upload.router)
```

### Test Backend

```bash
cd api
awsume permissions-platform-qa-generic
make dev

# In another terminal, test with curl:
curl -X POST http://localhost:8888/upload/ \
  -F "csv_files=@test.csv" \
  -F "manifest_file=@manifest.json" \
  -F "folder_path=test-upload" \
  -F "environment=qa"
```

---

## Phase 4: Update Frontend to Use Backend API ✅

### Goal

Replace direct S3 calls with HTTP requests to backend API.

### Files to Modify

#### 4.1: Create `frontend/src/services/uploadService.ts` ✅

Replace `s3Service.ts` with new service that calls backend:

```typescript
/**
 * Upload service - handles file uploads via backend API
 */
import {getUploadEndpoint} from './apiConfig';
import type {UploadResult} from '../types/backfill';
import type {UploadProgressCallback} from './s3Service';

/**
 * Upload files to backend API which forwards to S3
 */
export async function uploadFiles(
    csvFiles: File[],
    manifestFile: File | null,
    folderPath: string,
    environment: string,
    onProgress?: UploadProgressCallback,
): Promise<UploadResult> {
    const endpoint = getUploadEndpoint();

    try {
        // Create FormData
        const formData = new FormData();

        // Add CSV files
        csvFiles.forEach(file => {
            formData.append('csv_files', file);
        });

        // Add manifest file
        if (manifestFile) {
            formData.append('manifest_file', manifestFile);
        }

        // Add folder path and environment
        formData.append('folder_path', folderPath);
        formData.append('environment', environment);

        // Report progress (indeterminate since we can't track server-side progress)
        if (onProgress) {
            onProgress('Uploading files...', 1, 1);
        }

        // Send request
        const response = await fetch(endpoint, {
            method: 'POST',
            body: formData,
        });

        if (!response.ok) {
            const error = await response.json();
            throw new Error(error.detail || 'Upload failed');
        }

        const result = await response.json();

        return {
            status: result.status,
            message: result.message,
            uploadedFiles: result.uploaded_files,
            errors: result.errors,
        };

    } catch (error) {
        return {
            status: 'error',
            message: error instanceof Error ? error.message : 'Upload failed',
            errors: [error instanceof Error ? error.message : 'Unknown error'],
        };
    }
}

/**
 * Validate folder path (client-side validation, same as before)
 */
export function validateFolderPath(folderPath: string): string | null {
    const trimmed = folderPath.trim();

    if (!trimmed) {
        return 'Folder path cannot be empty';
    }

    if (trimmed.startsWith('/') || trimmed.endsWith('/')) {
        return 'Folder path should not start or end with a slash';
    }

    const reservedPaths = ['pdp_qa_refresh', 'integration-test'];
    if (reservedPaths.includes(trimmed)) {
        return `'${trimmed}' is a reserved folder path and cannot be used`;
    }

    const invalidChars = /[<>{}[\]\\^%`|"]/;
    if (invalidChars.test(trimmed)) {
        return 'Folder path contains invalid characters';
    }

    return null;
}
```

#### 4.2: Update `frontend/src/components/BackfillUploader/BackfillUploader.tsx` ✅

**Line 5 - Update import:**

```typescript
// OLD
import {uploadFiles, validateFolderPath} from '../../services/s3Service';
import {getBucketName} from '../../services/awsConfig';

// NEW
import {uploadFiles, validateFolderPath} from '../../services/uploadService';
```

**Line 44 - Remove bucket name logic:**

```typescript
// OLD
const bucket = getBucketName(environment);

// NEW (bucket is handled by backend)
// Remove this line entirely
```

**Line 88-104 - Update uploadFiles call:**

```typescript
// OLD
const result = await uploadFiles(
    selectedFiles.csvFiles,
    selectedFiles.manifestFile,
    folderPath,
    bucket,  // Remove bucket parameter
    (fileName, current, total) => {
    ...
    },
);

// NEW
const result = await uploadFiles(
    selectedFiles.csvFiles,
    selectedFiles.manifestFile,
    folderPath,
    environment,  // Pass environment instead of bucket
    (fileName, current, total) => {
    ...
    },
);
```

#### 4.3: Delete Old Frontend Service Files ✅

```bash
rm frontend/src/services/s3Service.ts
# awsConfig.ts was already deleted in Phase 2
```

---

## Phase 5: Docker Configuration for Monorepo ✅

### Goal

Update Docker setup to run both frontend and backend services together.

### Root `docker-compose.yml` ✅

Create new file at project root to orchestrate both services:

```yaml
version: '3.8'

services:
  backend:
    build:
      context: ./api
      dockerfile: Dockerfile
    container_name: pdp-backfill-backend
    ports:
      - "8888:8888"
    volumes:
      # Mount AWS credentials for awsume
      - ~/.aws:/root/.aws:ro
    environment:
      - Environment=dev
      - AWS_REGION=${AWS_REGION:-us-east-1}
      - QA_BUCKET=${QA_BUCKET:-qa-pdp-backfill}
      - PROD_BUCKET=${PROD_BUCKET:-prod-pdp-backfill}
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:8888/hello/" ]
      interval: 10s
      timeout: 5s
      retries: 5

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
      args:
        DOCKER_BUILDKIT: 1
      secrets:
        - GITHUB_NPM_TOKEN
    container_name: pdp-backfill-frontend
    ports:
      - "3000:3000"
    volumes:
      # Mount source for hot reload
      - ./frontend/src:/app/src
      - ./frontend/frontend.json:/app/frontend.json
      - ./frontend/tsconfig.json:/app/tsconfig.json
      - ./frontend/biome.json:/app/biome.json
      - /app/node_modules
    environment:
      - NODE_ENV=development
      - VITE_API_URL=http://backend:8888
    depends_on:
      backend:
        condition: service_healthy
    stdin_open: true
    tty: true

secrets:
  GITHUB_NPM_TOKEN:
    environment: 'GITHUB_NPM_TOKEN'
```

### Usage

```bash
# Start both services
docker-compose up --build

# Frontend: http://localhost:3000
# Backend: http://localhost:8888
# Backend docs: http://localhost:8888/docs
```

---

## Phase 6: Update Documentation ✅

### 6.1: Update `CLAUDE.md` ✅

**Updated sections:**

1. **Project Overview** - Added monorepo structure and backend proxy architecture explanation
2. **Development Commands** - Separated into Backend (FastAPI) and Frontend (React) sections
3. **Project Structure** - Updated to show complete monorepo layout with frontend/ and api/ directories
4. **Architecture Overview** - Documented backend proxy pattern and upload flow
5. **AWS Credentials Setup** - Removed VITE_ workflow, documented backend-only approach with awsume
6. **Working with Services** - Updated to describe frontend (apiConfig.ts, uploadService.ts) and backend services (s3_service.py, upload.py)

### 6.2: Update Root `README.md` ✅

Created comprehensive root README with:

- Project overview and monorepo structure
- ASCII art diagram showing frontend ↔ backend ↔ S3 interaction flow
- Quick start instructions for local and Docker development
- Environment variables documentation
- Development workflow for both frontend and backend
- Testing instructions
- Key features and technology stack

### 6.3: Update `_tasks/03-Troubleshoot-AWS-Connectivity-Issues.md` ✅

Already updated in Phase 2.2 with decision change note.

---

## Phase 7: Testing & Verification

### 7.1: Backend Testing

```bash
cd api
awsume permissions-platform-qa-generic

# Run unit tests
make test_unit

# Start dev server
make dev

# Test upload endpoint
curl -X POST http://localhost:8888/upload/ \
  -F "csv_files=@_tmp/backfill/test.csv" \
  -F "manifest_file=@_tmp/backfill/manifest.json" \
  -F "folder_path=test-upload-$(date +%s)" \
  -F "environment=qa"
```

### 7.2: Frontend Testing

```bash
cd frontend
pnpm install
pnpm check  # Lint, format check, typecheck

# Start dev server (backend must be running)
pnpm start

# Manual test:
# 1. Navigate to http://localhost:3000
# 2. Select QA environment
# 3. Choose CSV files and manifest
# 4. Enter folder path
# 5. Review and upload
# 6. Verify files appear in S3
```

### 7.3: Integration Testing (Docker)

```bash
# From project root
docker-compose up --build

# Test frontend at http://localhost:3000
# Check backend logs: docker logs pdp-backfill-backend
# Check frontend logs: docker logs pdp-backfill-frontend
```

### 7.4: S3 Verification

```bash
awsume permissions-platform-qa-generic

# List uploaded files
aws s3 ls s3://qa-pdp-backfill/test-upload-*/

# Download and verify
aws s3 cp s3://qa-pdp-backfill/test-upload-*/manifest.json ./test-manifest.json
cat test-manifest.json
```

---

## Rollback Plan

If something goes wrong, rollback steps:

### 1. Revert Git Commits

```bash
# Find commit before restructure
git log --oneline

# Revert to that commit
git reset --hard <commit-sha>
```

### 2. Restore Option 1 Implementation

If you need to temporarily restore direct browser-to-S3:

```bash
# Restore from git history
git checkout <option-1-commit> -- src/services/awsConfig.ts
git checkout <option-1-commit> -- src/services/s3Service.ts
git checkout <option-1-commit> -- package.json
```

### 3. Emergency CORS Fix

If you need uploads working immediately:

```bash
# Apply CORS config (see _tasks/03-Troubleshoot-AWS-Connectivity-Issues.md)
awsume permissions-platform-qa-generic
aws s3api put-bucket-cors --bucket qa-pdp-backfill --cors-configuration file://cors-config.json
```

---

## Success Criteria

- [ ] Project restructured into frontend/ and api/ directories
- [ ] All VITE credential code removed from frontend
- [ ] AWS SDK dependencies removed from frontend
- [ ] Backend upload endpoint implemented and tested
- [ ] Frontend successfully calls backend API
- [ ] Files upload to S3 via backend proxy
- [ ] No CORS errors in browser console
- [ ] Documentation updated (CLAUDE.md, README.md)
- [ ] Docker Compose works for both services
- [ ] Tests pass (backend unit tests)
- [ ] Manual E2E test successful (QA environment)

---

## Timeline Estimate

- Phase 1 (Restructure): 30-60 minutes
- Phase 2 (Rollback): 15-30 minutes
- Phase 3 (Backend Service): 2-3 hours
- Phase 4 (Frontend Updates): 1-2 hours
- Phase 5 (Docker): 30-60 minutes
- Phase 6 (Documentation): 1 hour
- Phase 7 (Testing): 1-2 hours

**Total: ~7-10 hours of implementation + testing**

---

## Notes

- Keep Option 1 code in git history (don't delete permanently)
- Test thoroughly in QA before touching Production
- Backend naturally reads AWS credentials from shell (awsume works perfectly)
- Consider adding rate limiting to backend upload endpoint in future
- Consider adding file size limits to prevent abuse