# PDP Backfill Application - Implementation Plan

## ⚠️ ARCHITECTURE CHANGE NOTICE (2026-01-07)

**This document describes the ORIGINAL implementation (Option 1: Direct Browser-to-S3 Uploads) which has been
SUPERSEDED.**

### What Changed

We have decided to switch from **Option 1** (direct browser-to-S3 uploads with VITE credentials) to **Option 2** (
FastAPI backend proxy architecture).

### Why

After implementing Option 1 and encountering CORS issues, we determined that a backend proxy architecture is superior:

- **No CORS configuration needed** on QA/Production S3 buckets
- **Better security** - AWS credentials never exposed to browser
- **Cleaner architecture** - Clear frontend/backend separation
- **Natural awsume integration** - Backend runs server-side where credentials work natively

### New Implementation Plan

**See:** `/Users/tcalhoun/work/collab/technicalelvis/pdp-backfill-app/_tasks/04-Create-Backend-Upload-Service.md`

The new plan includes:

- Restructuring project into `frontend/` and `api/` monorepo directories
- Rolling back VITE credential changes from this plan
- Implementing FastAPI backend upload service
- Updating frontend to call backend API instead of S3 directly

### Status of This Document

This document is **kept for historical reference** showing the original Option 1 implementation. The phases and
implementation details below describe what was completed with Option 1, which will be partially rolled back and replaced
with the backend proxy architecture.

---

## Overview

This document outlines the implementation plan for a single-page React application that simplifies the PDP (Permissions
Platform) backfill process. The app will replace manual AWS Console/CLI operations with a user-friendly interface for
uploading CSV and JSON files to S3.

**MVP Scope:** This is a simple local development tool that runs in a Docker container. No authentication, no production
deployment, just a straightforward file upload interface using local AWS credentials.

**NOTE:** The implementation described below uses direct browser-to-S3 uploads (Option 1), which is being replaced with
a backend proxy architecture (Option 2). See notice above.

## Implementation Notes

**Suite Components Usage (Discovered during Phase 2):**

The `@theorchard/suite-components` library (v12.30.1) has a more limited set of layout components than initially
planned:

- ✅ **Available:** `PageHeader`, `ToastProvider`, form components, buttons, modals, etc.
- ❌ **Not Available:** `SuiteProvider`, `Page`, `Box`, `Text`, `Link` layout components

**Approach:**

- Use `ToastProvider` for Suite integration
- Use `PageHeader` component with `mainContent` prop structure for page headers
- Use standard HTML elements (`div`, `p`, `h1-h6`, `a`, etc.) for layout
- Use custom CSS classes for styling and layout
- Suite's base styles provide typography and design tokens

**Biome Configuration:**

- Biome CLI v2.2.4 requires schema version 2.2.4 in `biome.json`
- Run `npx biome migrate --write` when schema version mismatches occur
- Key changes in v2.2.4:
    - `files.ignore` → `files.includes` with negation patterns
    - `organizeImports` → `assist.actions.source.organizeImports`
    - `overrides[].include` → `overrides[].includes`

## Project Structure

```
pdp-backfill-app/
├── Dockerfile                      # Docker container configuration
├── docker-compose.yml              # Docker compose setup
├── .env.example                    # Example environment variables
├── package.json                    # Dependencies and scripts
├── tsconfig.json                   # TypeScript configuration
├── biome.json                      # Biome linting/formatting config
├── frontend.json                   # Suite Frontend CLI configuration
├── .gitignore                      # Git ignore patterns
└── src/
    ├── index.tsx                   # Application entry point
    ├── app.tsx                     # Main app setup with Suite Frontend
    ├── styles/
    │   └── index.css              # Global styles
    ├── pages/
    │   └── home/
    │       ├── index.ts           # Export barrel
    │       └── HomePage.tsx       # Main page component
    ├── components/
    │   ├── BackfillUploader/
    │   │   ├── index.ts
    │   │   ├── BackfillUploader.tsx       # Main uploader component
    │   │   ├── FileSelector.tsx           # CSV/JSON file selection
    │   │   ├── FolderPathInput.tsx        # S3 folder path input
    │   │   ├── FileReview.tsx             # Review selected files
    │   │   └── UploadStatus.tsx           # Upload progress/status
    │   └── index.ts               # Component exports
    ├── services/
    │   ├── s3Service.ts           # AWS S3 upload operations
    │   └── awsConfig.ts           # AWS SDK configuration
    ├── types/
    │   ├── index.ts               # Type exports
    │   ├── backfill.ts            # Backfill-related types
    │   └── manifest.ts            # Manifest file types
    └── schemas/
        ├── index.ts               # Schema exports
        └── manifestSchema.ts      # Zod schema for manifest validation
```

## Technology Stack

### Core Framework

- **React 18.3.1** - UI library with TypeScript
- **TypeScript 5.9+** - Type safety and developer experience
- **React Router v5** - Client-side routing

### UI Components

- **@theorchard/suite-components** (v12.30.1) - Suite UI component library
- **@theorchard/suite-frontend** (v8.10.1) - Suite application framework
- **@theorchard/suite-icons** (v7.15.0) - Icon library

### Build & Development

- **Vite** - Fast build tool (via @theorchard/frontend-cli-vite)
- **@theorchard/frontend-cli** (v4.0.1) - Suite frontend build system

### AWS Integration

- **@aws-sdk/client-s3** (v3.x) - S3 operations (upload files)
- **@aws-sdk/credential-providers** (v3.x) - AWS credential management

### Validation & Formatting

- **Zod** (v4.x) - Runtime type validation for file contents
- **Biome** (v2.2.4) - Fast linting and formatting

### Testing (Future)

- **@theorchard/suite-testing** - Testing utilities
- **Vitest** - Unit testing framework

## Implementation Phases

### Phase 1: Project Setup & Configuration ✅

#### 1.1 Package Configuration

Create `package.json` with:

- All required dependencies from tech stack
- Scripts for:
    - `start` - Development server
    - `build` - Production build
    - `lint` - Run Biome linting
    - `lint:fix` - Auto-fix linting issues
    - `format` - Format code
    - `typecheck` - TypeScript type checking
    - `check` - Run all checks (lint + format + typecheck)

#### 1.2 TypeScript Configuration

Create `tsconfig.json` matching reference project:

- Strict mode enabled
- React JSX configuration
- Path aliases for `src/*`
- Target ES2020+
- Module resolution: bundler

#### 1.3 Biome Configuration

Create `biome.json` based on reference project:

- Formatting rules (4 spaces, single quotes, semicolons)
- Linting rules (no default exports except for lazy loading)
- Import organization (React imports first)
- File includes/excludes

#### 1.4 Frontend CLI Configuration

Create `frontend.json` for Suite Frontend:

- Configure HTML generation
- Set app title: "PDP Backfill Upload"
- Configure environment variable placeholders (simplified for local use)
- Set up Vite plugin
- **No authentication configuration needed**

#### 1.5 Docker Configuration

Create `Dockerfile`:

- Base image: Node 18 or 20
- Install dependencies
- Run development server on port 3000
- Mount source code as volume for hot reload

Create `docker-compose.yml`:

- Map port 3000:3000
- Mount AWS credentials from host (~/.aws)
- Mount source code for development
- Set environment variables

Create `.env.example`:

- AWS_REGION=us-east-1
- AWS_PROFILE=permissions-platform-qa-generic (or prod)
- Document required AWS setup

#### 1.6 Git Configuration

Update `.gitignore`:

- node_modules
- dist/build output
- .cache
- .env files (keep .env.example)
- IDE-specific files

### Phase 2: Application Foundation ✅

#### 2.1 Entry Point (`src/index.tsx`) ✅

**Implemented:**

- Imports React DOM's `createRoot` for React 18
- Imports App component and global styles
- Renders app to DOM using root element

#### 2.2 App Configuration (`src/app.tsx`) ✅

**Implemented:**

- Uses `ToastProvider` from `@theorchard/suite-components` for toast notifications
- Sets up React Router v5 with BrowserRouter
- **Simplified setup:** No authentication, no complex navigation
- Single route to HomePage at path "/"
- Minimal single page app architecture

**Note:** Suite components library doesn't include `SuiteProvider` - used `ToastProvider` instead for basic Suite
integration.

#### 2.3 Home Page (`src/pages/home/HomePage.tsx`) ✅

**Implemented:**

- Uses Suite `PageHeader` component with `mainContent` prop structure
- Uses standard HTML elements (`div`, `p`, `ol`, `a`) for layout instead of Suite layout components
- Includes descriptive text about the PDP backfill process
- Shows getting started steps in an ordered list
- Placeholder for `BackfillUploader` component (Phase 5)
- Helpful links to PDP documentation and Slack channel
- Custom CSS classes for styling: `.app-container`, `.page-content`, `.section`, `.placeholder-box`

**Note:** Suite components library doesn't include `Page`, `Box`, `Text`, or `Link` layout components - used standard
HTML elements with custom CSS classes.

#### 2.4 Global Styles (`src/styles/index.css`) ✅

**Implemented:**

- Imports Suite base styles from `@theorchard/suite-components/dist/index.css`
- Added app layout styles (`.app-container`, `.page-content`)
- Added section styling with card-like appearance
- Added placeholder box styling for future component
- Includes custom styles for upload interface (for Phase 5)
- Styles for file preview sections
- Styles for status messages (success, error, warning, info)
- Progress indicator styles
- Responsive adjustments for mobile

#### 2.5 Configuration Updates ✅

**Biome Configuration Migration:**

- Migrated `biome.json` schema from v1.9.4 to v2.2.4
- Changed `files.ignore` to `files.includes` with negation patterns
- Moved `organizeImports` to `assist.actions.source.organizeImports`
- Updated `overrides[].include` to `overrides[].includes`

### Phase 3: Type Definitions & Schemas ✅

#### 3.1 Type Definitions (`src/types/`)

**backfill.ts:**

```typescript
// S3 bucket configuration
export type Environment = 'qa' | 'prod';

export interface S3Config {
    bucket: string;
    region: string;
}

// File upload state
export interface SelectedFiles {
    csvFiles: File[];
    manifestFile: File | null;
}

// Upload status
export type UploadStatus = 'idle' | 'uploading' | 'success' | 'error';

export interface UploadResult {
    status: UploadStatus;
    message?: string;
    uploadedFiles?: string[];
    errors?: string[];
}
```

**manifest.ts:**

```typescript
// Manifest file structure
export interface ManifestJob {
    job_type: 'attach_and_detach';
    keys: string[];
}

export interface Manifest {
    bucket: string;
    jobs: ManifestJob[];
}
```

#### 3.2 Validation Schemas (`src/schemas/`)

**manifestSchema.ts:**

```typescript
import {z} from 'zod';

export const manifestJobSchema = z.object({
    job_type: z.literal('attach_and_detach'),
    keys: z.array(z.string()).min(1, 'At least one CSV key required'),
});

export const manifestSchema = z.object({
    bucket: z.string().min(1, 'Bucket name required'),
    jobs: z.array(manifestJobSchema).min(1, 'At least one job required'),
});

export type ValidatedManifest = z.infer<typeof manifestSchema>;
```

Note: CSV validation schemas will be added in a future enhancement phase.

### Phase 4: AWS S3 Integration ✅

**⚠️ DEPRECATED:** This phase describes the Option 1 implementation which is being replaced. The new backend proxy
architecture (Option 2) moves all AWS/S3 logic to the backend. See task 04 for the new approach.

#### 4.1 AWS Configuration (`src/services/awsConfig.ts`) ✅

```typescript
// S3 bucket names based on environment
// AWS region configuration from environment variables
// Use AWS SDK default credential provider chain
```

Key considerations (Option 1 - being replaced):

- **Local Development:** AWS credentials mounted from host machine's ~/.aws directory
- User must have AWS CLI configured with appropriate profile
- Use environment variables: AWS_REGION, AWS_PROFILE
- Buckets: `qa-pdp-backfill` and `prod-pdp-backfill`
- Region: `us-east-1` (default)
- AWS SDK will automatically use credentials from the Docker container's mounted ~/.aws

#### 4.2 S3 Service (`src/services/s3Service.ts`) ✅

Functions implemented:

- `uploadFile(file: File, s3Path: string, bucket: string)` - Upload single file
- `uploadFiles(files: File[], folderPath: string, bucket: string)` - Upload multiple files
- `validateCredentials()` - Check AWS credentials are valid
- Error handling for:
    - Network failures
    - Permission errors (403)
    - Invalid credentials
    - File type restrictions

Upload strategy:

1. Upload all CSV files first
2. Wait for all CSVs to complete
3. Only then upload manifest.json (which triggers the backfill)
4. Provide progress feedback for each file

### Phase 5: UI Components ✅

**Component Development Approach:**

- Use Suite components where available (buttons, inputs, modals, forms, etc.)
- Use standard HTML elements with custom CSS classes for layout
- Leverage Suite's design tokens and base styles
- Reference Suite components documentation for available components

**Implementation Notes:**

- All components implemented with TypeScript and React hooks
- Uses Suite Button, Input, and Spinner components
- Custom CSS for layout and styling
- Comprehensive validation and error handling
- Step-by-step workflow with clear navigation

#### 5.1 BackfillUploader Component ✅

**Main orchestrator component:**

Implemented features:

- State management for:
    - Selected CSV files
    - Selected manifest file
    - Folder path input
    - Environment selection (QA/Prod)
    - Upload status and progress
- Step-by-step workflow:
    1. Environment selection (QA/Prod buttons)
    2. File selection (CSV and manifest)
    3. Folder path input (with validation)
    4. File review (with manifest validation)
    5. Upload execution (with progress tracking)
    6. Status display (success/error states)
- Integrated with s3Service for file uploads
- Progress callback for real-time upload updates

#### 5.2 FileSelector Component ✅

**Purpose:** File selection interface

Implemented features:

- Multiple CSV file selector (accept: `.csv`)
- Single JSON file selector (accept: `.json`)
- File validation:
    - Check file extensions
    - Validate JSON can be parsed
    - Display file names and sizes
- Remove individual files
- File size formatting (B, KB, MB)
- Disabled state support

#### 5.3 FolderPathInput Component ✅

**Purpose:** S3 folder path configuration

Implemented features:

- Text input for folder path (e.g., "PP-1055")
- Real-time validation using validateFolderPath from s3Service:
    - No leading/trailing slashes
    - Valid S3 key characters only
    - Blocked reserved paths (pdp_qa_refresh, integration-test)
- Helpful placeholder and guidelines
- Preview of final S3 paths
- Warning box about reserved folders
- Disabled state support

#### 5.4 FileReview Component ✅

**Purpose:** Review screen before upload

Implemented features:

- Display environment (QA/Prod) and bucket name
- List all CSV files with full S3 paths
- Show manifest file with S3 path
- Expandable manifest content viewer (show/hide)
- Manifest validation with Zod schema:
    - Structure validation
    - Keys match CSV filenames
    - Bucket name matches environment
- Warning messages:
    - CSV files uploaded before manifest
    - Manifest triggers backfill immediately
    - Extra warning for production environment
- Action buttons:
    - "Submit Backfill" (primary, disabled if invalid)
    - "Go Back" (secondary)

#### 5.5 UploadStatus Component ✅

**Purpose:** Display upload progress and results

Implemented features:

- Loading state with Spinner:
    - "Uploading CSV files... (current/total)"
    - Current file name
    - "Uploading manifest.json..." with warning
- Success state:
    - Checkmark icon (✓)
    - "Backfill submitted successfully!"
    - List of uploaded files with S3 paths
    - Links to Datadog dashboard and Slack channel
    - "Upload Another Backfill" button
- Error state:
    - Error icon (✕)
    - Detailed error message
    - List of error details
    - Partial success indicator (files uploaded before error)
    - Resolution suggestions
    - Links to support channels
    - "Try Again" button

### Phase 6: Component Implementation Details ✅

**Phase 6 Implementation Notes:**
All implementation details from Phase 5 already address Phase 6 requirements. Enhanced with accessibility improvements
and comprehensive state management.

#### State Management ✅

Implemented using React hooks:

- **`useState`** - Used throughout all components for:
    - Form values (folderPath, environment selection)
    - File selections (csvFiles, manifestFile)
    - Upload status (isUploading, uploadResult, uploadProgress)
    - UI state (currentStep, showManifest, manifestValid)
- **No `useReducer` needed** - The workflow state is simple enough for useState
- **No Context API needed** - Props are sufficient for this single-page app
- All state is properly typed with TypeScript interfaces

#### Error Handling Strategy ✅

Comprehensive error handling implemented:

1. **File validation errors** ✅ - Display inline in FileSelector component
    - CSV extension validation with alerts
    - JSON parsing validation with error messages
    - File size display for user awareness

2. **AWS credential errors** ✅ - Handled in s3Service with specific error types
    - S3UploadError class for structured error handling
    - validateCredentials function checks access before upload
    - Clear error messages displayed in UploadStatus component

3. **Upload errors** ✅ - Shown in UploadStatus component with:
    - Detailed error messages
    - Error icon and styling
    - Resolution suggestions
    - Support contact links

4. **Partial failures** ✅ - UploadStatus shows:
    - Which files uploaded successfully before error
    - Separate list of errors
    - Clear indication of partial success state

5. **Network errors** - Basic error handling implemented
    - Future enhancement: Retry logic with exponential backoff

#### User Experience Flow ✅

Complete step-by-step workflow implemented:

```
1. Select Environment (QA/Prod) ✅
   → Button selection loads appropriate bucket config
   → Environment displayed throughout workflow

2. Select CSV Files ✅
   → Validates .csv extensions
   → Shows file list with sizes
   → Remove individual files option

3. Select Manifest JSON ✅
   → Validates .json extension
   → Parses JSON and validates structure
   → Shows file with size

4. Enter Folder Path ✅
   → Real-time validation
   → Shows preview of final S3 keys
   → Blocks reserved paths

5. Review Screen ✅
   → Shows all files with full S3 paths
   → Displays environment and bucket
   → Expandable manifest content
   → Manifest validation with Zod
   → Submit/Go Back buttons

6. Upload ✅
   → Upload CSVs with progress (current/total)
   → Shows current file being uploaded
   → Waits for completion
   → Uploads manifest with warning
   → Shows success/error with details

7. Complete ✅
   → Success: Links to monitoring, upload another option
   → Error: Retry option, support links
```

#### Accessibility Considerations ✅

Comprehensive accessibility features implemented:

1. **ARIA labels** ✅
    - File inputs have aria-label and aria-describedby
    - Environment buttons have descriptive aria-labels
    - Sections use role="region" with aria-labelledby

2. **Keyboard navigation** ✅
    - All interactive elements keyboard accessible
    - Focus indicators with visible outlines (3px solid blue)
    - Focus-visible pseudo-class for modern browser support

3. **Screen reader support** ✅
    - Error messages use role="alert" and aria-live="assertive"
    - Loading states use role="status" and aria-live="polite"
    - Success/error states use role="alert" and aria-live="assertive"
    - Visually hidden text for additional context
    - Icons marked with aria-hidden="true"

4. **Loading states** ✅
    - Spinner component with role="status"
    - Progress text announced to screen readers
    - Clear indication of upload stages

5. **Visual + semantic indicators** ✅
    - Icons (✓ and ✕) combined with text
    - Color coding combined with text labels
    - Clear headings and semantic HTML structure

6. **Additional features** ✅
    - .visually-hidden CSS class for screen reader-only text
    - Skip-to-main link support in CSS (for future use)
    - Proper heading hierarchy throughout

### Phase 7: Validation & Safety Features ✅

#### Pre-Upload Validations ✅

1. **CSV Files:** ✅
    - Must have `.csv` extension
    - Basic file validation only (detailed CSV content validation is a future enhancement)

2. **Manifest File:** ✅
    - Must be valid JSON
    - Must match manifest schema
    - Keys in manifest must reference uploaded CSV filenames
    - Bucket name must match selected environment

3. **Folder Path:** ✅
    - Cannot be `pdp_qa_refresh/` (reserved)
    - Cannot be `integration-test/` (reserved)
    - Must be valid S3 key prefix

#### Upload Safeguards ✅

1. ✅ Upload CSVs first, manifest last (prevents premature trigger)
2. ✅ Confirm all CSVs uploaded before proceeding to manifest
3. ✅ Show clear warning before manifest upload
4. ✅ Production environment requires additional confirmation (modal with detailed warning)
5. ✅ Display estimated time for upload based on file sizes

#### Post-Upload Actions ✅

1. ✅ Display Datadog dashboard link:
    - QA: Link to QA backfill dashboard
    - Prod: Link to Prod backfill dashboard
2. ✅ Instructions for monitoring progress
3. ✅ Link to Slack channel (#permissions-platform-public)
4. ✅ Clear success confirmation

**Implementation Details:**

- **Production Confirmation Modal** (BackfillUploader.tsx): Shows detailed warning with checklist before production
  uploads
- **Environment-specific Datadog Links** (UploadStatus.tsx): Separate dashboard URLs for QA and Prod environments
- **Estimated Upload Time** (FileReview.tsx): Calculates and displays estimated time based on total file sizes (assumes
  1 MB/s upload speed)
- **Environment Variables** (.env.example): Updated with QA and Prod Datadog dashboard URLs

### Phase 8: Environment Configuration ✅

**⚠️ PARTIALLY DEPRECATED:** Parts of this phase (VITE_AWS credentials) are being rolled back in the Option 2
implementation. The backend proxy will handle AWS credentials server-side.

#### Environment Variables (.env file) ✅

**Implemented (Option 1 - being partially rolled back):**

- Updated `.env` and `.env.example` with comprehensive environment variable configuration
- Added VITE_-prefixed variables for browser accessibility:
    - ~~`VITE_AWS_REGION` - AWS region configuration~~ (being removed - backend only)
    - ~~`VITE_AWS_ACCESS_KEY_ID`, `VITE_AWS_SECRET_ACCESS_KEY`, `VITE_AWS_SESSION_TOKEN`~~ (being removed - backend
      only)
    - `VITE_QA_BUCKET` and `VITE_PROD_BUCKET` - S3 bucket names (keeping for display purposes)
    - `VITE_DATADOG_QA_DASHBOARD_URL` and `VITE_DATADOG_PROD_DASHBOARD_URL` - Environment-specific monitoring
      dashboards (keeping)
    - `VITE_SLACK_CHANNEL_URL` - Support channel link (keeping)
    - **NEW in Option 2:** `VITE_API_URL` - Backend API endpoint
- Non-VITE_ prefixed versions for Docker container environment (AWS_REGION, AWS_PROFILE, etc.)
- Comprehensive inline documentation explaining variable purposes

#### AWS Credentials (Local Docker Setup) ✅

**Implemented (Option 1 - being replaced):**

- ✅ ~/.aws directory mounted from host to container in docker-compose.yml (moving to backend container only)
- ✅ AWS_PROFILE environment variable configured with default value (backend only)
- ~~✅ AWS SDK automatically uses credentials from mounted directory~~ (frontend no longer uses AWS SDK)
- ~~✅ Enhanced error handling in s3Service.ts with specific messages~~ (moving to backend API)

#### Configuration Updates ✅

**Implemented:**

- Updated `awsConfig.ts` to use VITE_QA_BUCKET and VITE_PROD_BUCKET environment variables (with fallback defaults)
- Updated `UploadStatus.tsx` to use environment variables for Datadog and Slack URLs
- Added TypeScript type definitions for all new environment variables in `vite-env.d.ts`
- All environment variables have sensible fallback defaults to prevent runtime errors

**Files Modified:**

- `.env` - Added VITE_-prefixed variables
- `.env.example` - Updated to match with better documentation
- `src/vite-env.d.ts` - Added TypeScript types for new environment variables
- `src/services/awsConfig.ts` - Use environment variables for bucket configuration
- `src/components/BackfillUploader/UploadStatus.tsx` - Use environment variables for external links
- `src/services/s3Service.ts` - Enhanced error messages for expired credentials

### Phase 9: Documentation & Help

#### In-App Help

1. **Tooltips:** Explain each field (folder path, environment selection)
2. **Info Cards:** Link to full backfill documentation
3. **Examples:** Show example folder paths, CSV formats
4. **Video Link:** Embed link to demo video from documentation

#### README Updates

- **Prerequisites:** Docker, Docker Compose, AWS CLI configured
- **AWS Setup:** How to configure AWS CLI with `generic-engineer-role`
- **Running locally:** `docker-compose up` command
- **Accessing the app:** http://localhost:3000
- **Stopping the app:** `docker-compose down`
- **Troubleshooting common issues:** AWS credentials, permissions, S3 access

### Phase 10: Testing Strategy

#### Manual Testing Checklist

- [ ] Select multiple CSV files
- [ ] Select manifest JSON file
- [ ] Invalid file types rejected
- [ ] Invalid manifest JSON shows error
- [ ] Folder path validation works
- [ ] CSV uploaded before manifest
- [ ] Success state displays correctly
- [ ] Error states display correctly
- [ ] Can upload another batch after success
- [ ] Works in a local docker environment

#### Future Automated Tests

- Unit tests for validation functions
- Unit tests for S3 service (mocked)
- Integration tests for file upload flow
- E2E tests for full workflow

## Implementation Order

1. **Setup** (Phase 1): Configuration files, dependencies, Docker setup
2. **Foundation** (Phase 2): App structure, routing, basic layout
3. **Types** (Phase 3): TypeScript types and Zod schemas
4. **AWS** (Phase 4): S3 service and AWS configuration
5. **Components** (Phase 5): Build UI components
6. **Validation** (Phase 7): Add validation and safety features
7. **Polish** (Phase 6): Improve UX, error handling, accessibility
8. **Docker** (Phase 8): Finalize Docker setup and environment config
9. **Documentation** (Phase 9): In-app help and README with Docker instructions
10. **Testing** (Phase 10): Manual testing in Docker container

## Open Questions & Decisions Needed

1. **AWS Region:**
    - Confirm S3 bucket region is us-east-1
    - Should region be configurable via environment variable?

2. **File Size Limits:**
    - Maximum CSV file size?
    - Maximum number of CSV files per upload?
    - Show warning for large files?

3. **Production Safeguards:**
    - Extra confirmation modal for prod uploads?
    - Require entering "CONFIRM" to proceed in prod?

4. **Docker Image:**
    - Base Node image version (18 vs 20)?
    - Development mode only or also build production image?

## Success Criteria

**MVP Requirements:**

- ✅ Runs locally in Docker container
- ✅ Users can upload CSV and JSON files to S3
- ✅ App validates files before upload (basic validation)
- ✅ CSVs uploaded before manifest (prevents premature trigger)
- ✅ Clear success/error feedback
- ✅ Works with both QA and Prod S3 buckets
- ✅ Uses local AWS credentials (no authentication needed)
- ✅ Follows Suite UI design patterns
- ✅ Type-safe with TypeScript and Zod
- ✅ Clean code passing Biome linting
- ✅ Simple to start: `docker-compose up`

## Future Enhancements

**Beyond MVP:**

1. **CSV Validation:** Implement Zod schemas to validate CSV content
    - Check required headers (identity_uuid, tenant_uuid, tenant_type, role, operation)
    - Validate UUIDs are properly formatted
    - Validate tenant_type is one of: account, company_brand, label
    - Validate operation is either attach or detach
    - Show preview of CSV data with validation errors highlighted

2. **Authentication & Authorization:**
    - Add Auth0 or similar authentication
    - User login/logout functionality
    - Role-based access control
    - Audit logging of who uploaded what

3. **Production Deployment:**
    - Deploy to QA/Prod environments
    - CI/CD pipeline
    - Environment-specific configurations
    - Monitoring and alerting

4. **History:** View past uploads for current user

5. **Templates:** Save/load folder path + environment presets

6. **CSV Editor:** Edit CSV data in-app before upload

7. **Manifest Generator:** Auto-generate manifest from selected CSVs

8. **Validation Service:** Backend API for deeper validation

9. **Notifications:** Email/Slack notification when backfill completes

10. **Dry Run:** Preview what would happen without uploading

11. **Rollback:** Delete uploaded files if manifest upload fails

---

## Next Steps (Architecture Migration)

This document represents the **completed Option 1 implementation** (direct browser-to-S3 uploads).

**Current Status:** We are now migrating to **Option 2** (FastAPI backend proxy architecture).

### Migration Plan

**See full migration plan:** `_tasks/04-Create-Backend-Upload-Service.md`

**Summary of changes:**

1. **Restructure** - Move frontend files to `frontend/` directory (monorepo)
2. **Rollback** - Remove VITE credential code and AWS SDK from frontend
3. **Backend** - Implement FastAPI upload service with boto3
4. **Frontend** - Update to call backend API instead of S3 directly
5. **Docker** - Update docker-compose.yml for both services
6. **Documentation** - Update CLAUDE.md and README.md

### Why This Change

- **No CORS issues** - Backend-to-S3 communication bypasses browser CORS
- **Better security** - Credentials never exposed to browser
- **Cleaner architecture** - Clear separation of concerns
- **awsume works natively** - Backend runs server-side where credentials are accessible

### Historical Value

This document remains valuable as:

- **Reference** for the completed frontend UI implementation
- **Component documentation** - UI components, workflows, and UX patterns remain largely the same
- **Validation logic** - Much of the validation logic will be ported to backend
- **Historical context** - Shows the evolution of the architecture