# Architecture & Technical Details

Technical overview of the LocalStack development environment.

## Directory Structure

```
localstack/
├── terraform/                     # Local Terraform configurations
│   ├── _shared/                  # Shared configurations
│   │   └── provider_override.tf # LocalStack provider setup
│   └── <your-projects>/          # Your Terraform project directories
│       ├── main.tf               # Main resource definitions
│       ├── variables.tf          # Input variables
│       ├── outputs.tf            # Output values
│       └── _override.tf          # Symlink to _shared/provider_override.tf (auto-created)
├── scripts/                      # Deployment automation
│   ├── awslocal.sh              # AWS CLI wrapper
│   ├── check-prerequisites.sh   # Environment validation
│   ├── clean.sh                 # Interactive cleanup
│   ├── deploy-all.sh            # Deploy all projects
│   └── deploy-project.sh        # Deploy single project
├── docs/                         # Documentation
│   ├── WORKFLOWS.md             # Common workflows
│   ├── TROUBLESHOOTING.md       # Troubleshooting guide
│   ├── REFERENCE.md             # Commands and config
│   └── ARCHITECTURE.md          # This file
├── docker-compose.yaml           # LocalStack container config
├── Makefile                      # Build and deployment commands
├── .env.shadow                   # Environment template (tracked)
├── .env                          # Environment config (gitignored)
├── .gitignore                    # Git ignore rules
└── volume/                       # LocalStack persistent data (gitignored)
```

## Component Architecture

### LocalStack Container

**Purpose:** Provides local AWS service emulation

**Configuration:** `docker-compose.yaml`

**Services Enabled:**
- S3 - Object storage
- Lambda - Function execution
- Step Functions - Workflow orchestration
- EventBridge - Event routing
- IAM - Identity and access management
- STS - Security token service
- CloudWatch Logs - Logging
- ECR - Container registry
- SNS - Notifications

**Key Features:**
- Single endpoint: `http://localhost:4566`
- Persistence enabled (survives container restart)
- Docker-in-Docker for Lambda execution
- Volume mounted at `./volume` for data storage

### Terraform Projects

#### Shared Configuration

**Location:** `terraform/_shared/provider_override.tf`

**Purpose:** Overrides Terraform AWS provider to use LocalStack

**Distribution:** Symlinked to each project as `_override.tf` by `make tf-init`

### Example Projects

The following sections describe example projects that demonstrate the LocalStack setup.

#### Example: Lambda Projects

##### Example: lambda-av-scan

**Purpose:** Stub Lambda for virus scanning

**Resources:**
- Lambda function: `local-lambda-av-scan-containerized`
- IAM role for Lambda execution
- CloudWatch log group

**Behavior:**
- Always returns `{"status": "CLEAN"}`
- Simulates successful virus scan
- No actual scanning logic

**Why Stubs?**
- Fast deployment (no dependencies to package)
- Simple testing of workflow orchestration
- Focus on integration rather than implementation

#### Example: Workflow Project

**Purpose:** Example workflow configuration (e.g., ows-royalties-workflows)

**Resources:**

1. **S3 Buckets:**
   - `local-abacus-flowthrough` - Main upload bucket
   - `local-abacus-adjustments` - Adjustment files
   - etc.

2. **SNS Topics:**
   - `local-abacus-file-upload-events` - File upload notifications

3. **Step Functions:**
   - `local-file-upload-workflow` - Orchestrates:
     1. AV scan (calls lambda-av-scan)
     2. File processing (calls lambda-abacus)

4. **EventBridge Rules:**
   - Triggers Step Function on S3 upload to `uploads/` prefix
   - Event pattern matches: `s3:ObjectCreated:*`

**Data Flow:**
```
User uploads file to S3
    ↓
S3 sends event to EventBridge
    ↓
EventBridge triggers Step Function
    ↓
Step Function executes:
    1. Invoke AV Scan Lambda
    2. If clean, invoke File Processing Lambda
    ↓
File processing completes
```

## Differences from Production

This local environment has intentional simplifications:

### 1. No External Modules

**Production:**
```hcl
module "s3_bucket" {
  source = "git::https://github.com/..."
}
```

**Local:**
```hcl
resource "aws_s3_bucket" "bucket" {
  # Direct resource definition
}
```

**Why:** Simplifies dependencies and deployment speed

### 2. Stub Lambda Functions

**Production:**
- Full application code
- Multiple dependencies
- External API calls
- Database connections

**Local:**
- Minimal stub code
- No dependencies
- Returns mock success
- No external connections

**Why:** Focus on workflow orchestration, not implementation

### 3. Simplified IAM

**Production:**
- Cross-account roles
- Complex permission boundaries
- Service control policies
- Fine-grained permissions

**Local:**
- Basic execution roles
- Minimal required permissions
- No cross-account complexity

**Why:** LocalStack doesn't enforce IAM, minimal config needed

### 4. No KMS Encryption

**Production:**
```hcl
server_side_encryption_configuration {
  rule {
    apply_server_side_encryption_by_default {
      kms_master_key_id = aws_kms_key.key.arn
      sse_algorithm     = "aws:kms"
    }
  }
}
```

**Local:**
```hcl
server_side_encryption_configuration {
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}
```

**Why:** LocalStack KMS support is limited, AES256 sufficient for local testing

### 5. No Lifecycle Policies

**Production:**
- Transition to Glacier after 90 days
- Expire after 365 days
- Versioning with expiration rules

**Local:**
- No lifecycle rules
- Manual cleanup with `make clean`

**Why:** Local development is ephemeral, no need for archival

### 6. Local State Backend

**Production:**
```hcl
backend "s3" {
  bucket = "terraform-state-prod"
  key    = "qa/ows-royalties/terraform.tfstate"
  region = "us-east-1"
}
```

**Local:**
```hcl
backend "local" {
  path = "terraform.tfstate"
}
```

**Why:** No need for remote state in local development

## Provider Override Mechanism

### How It Works

1. **Shared Override File:**
   - Located at `terraform/_shared/provider_override.tf`
   - Contains LocalStack configuration
   - Tracked in git

2. **Per-Project Symlink:**
   - Each project gets `_override.tf` symlink
   - Points to shared file
   - Created by `make tf-init`

3. **Terraform Merge Behavior:**
   - Terraform merges all `.tf` files
   - `_override.tf` overrides default provider config
   - Allows using production Terraform with local changes

### Why Symlinks?

- **Single source of truth**: Update one file, affects all projects
- **No duplication**: Don't copy/paste override config
- **Version control**: Only one file to track
- **Easy cleanup**: Remove symlinks without touching source

## Deployment Dependencies

Projects must be deployed in order due to resource dependencies:

```
Example deployment order:
1. Lambda dependencies (e.g., lambda-av-scan, lambda-abacus)
   ↓ (provides Lambda functions)
2. Main workflows (e.g., ows-royalties-workflows)
   (references Lambda functions from step 1)
```

**Note:** Edit `scripts/deploy-all.sh` to customize deployment order for your projects.

**Enforced by:** `make deploy-all` and `scripts/deploy-all.sh`

## Persistence

### What Persists

LocalStack data persists across container restarts (but not `make clean`):

- S3 bucket contents
- Lambda functions
- Step Functions definitions
- EventBridge rules
- CloudWatch logs

**Storage:** `./volume/` directory

### What Doesn't Persist

- Terraform state (local files, not in container)
- Configuration changes to `docker-compose.yaml`
- Environment variables in `.env`

### Clean Slate

```bash
make clean  # Removes ALL LocalStack data and Terraform state
```

## Networking

### Container Network

**Network:** `localstack-network` (bridge mode)

**Allows:**
- LocalStack container to spawn Lambda containers
- Lambda containers to communicate with LocalStack services
- Host machine to access LocalStack on `localhost:4566`

### Port Mappings

```yaml
ports:
  - "4566:4566"            # Main LocalStack gateway
  - "4510-4559:4510-4559"  # External service ports (if needed)
```

**Why port range?** Some LocalStack services may need dedicated ports

## Resource Naming Convention

All resources use `local-` prefix for clarity:

```hcl
resource "aws_s3_bucket" "example" {
  bucket = "local-${var.name}"
}
```

**Benefits:**
- Clear distinction from production resources
- Easy to identify in AWS CLI output
- Prevents confusion when switching contexts

## Security Considerations

### Test Credentials

```bash
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test
```

**Safe because:**
- Only work with LocalStack
- Never touch real AWS
- Industry standard for LocalStack

### No Real AWS Access

The provider override ensures:
- All endpoints point to `localhost:4566`
- No possibility of touching real AWS
- Safe to experiment without fear

### Docker Socket Access

```yaml
volumes:
  - "/var/run/docker.sock:/var/run/docker.sock"
```

**Purpose:** Allows LocalStack to spawn Lambda containers

**Risk:** Container has Docker host access

**Mitigation:** Only for local development, not production

## Performance Characteristics

### First Deploy

- **Time:** 1-3 minutes
- **Why:** LocalStack initializing services, pulling Lambda images

### Subsequent Deploys

- **Time:** 10-30 seconds
- **Why:** Services already initialized, containers cached

### Resource Usage

**Typical:**
- CPU: 1-2 cores
- Memory: 2-4 GB
- Disk: 5-10 GB

**Under Load:**
- CPU: 2-4 cores
- Memory: 4-8 GB
- Disk: 10-20 GB

## Limitations

### LocalStack Community Edition

Some features are Pro-only:
- Advanced IAM policies
- Some CloudFormation features
- Advanced Lambda features
- RDS, ECS, EKS

**Impact:** Minimal for this project (we don't use Pro features)

### Not Perfect Emulation

LocalStack approximates AWS behavior but:
- Some edge cases differ
- Performance characteristics differ
- Some error messages differ

**Best Practice:** Test critical paths in real AWS (QA/staging) before production

## Extending the Environment

### Adding New Projects

1. Create directory:
   ```bash
   mkdir terraform/my-new-project
   ```

2. Add Terraform files:
   ```bash
   cd terraform/my-new-project
   # Create main.tf, variables.tf, outputs.tf
   ```

3. Initialize and deploy:
   ```bash
   make tf-init PROJECT=my-new-project
   make tf-apply PROJECT=my-new-project
   ```

The provider override is linked automatically.

### Adding New Services

Edit `docker-compose.yaml`:

```yaml
environment:
  - SERVICES=s3,lambda,stepfunctions,events,iam,sts,logs,ecr,cloudwatch,sns,sqs,dynamodb
  #                                                                            ^^^ add here
```

Then restart:
```bash
make stop
make start
```

### Adding Scripts

Add to `scripts/` directory:
```bash
touch scripts/my-script.sh
chmod +x scripts/my-script.sh
```

Update documentation in `docs/REFERENCE.md`.

## Future Enhancements

Potential improvements:
- CI/CD integration (GitHub Actions)
- Automated testing of workflows
- Pre-commit hooks for Terraform validation
- Docker Compose profiles for different service sets
- Integration with actual Lambda code repositories