sendgrid-webhooks
==================

## Overview
An AWS Lambda handler (invoked via an AWS Application Load Balancer - ALB) that processes and validates SendGrid event webhook requests. It receives webhook notifications from SendGrid for email events (processed, delivered, opened, clicked, bounced, etc.), validates the authenticity of the requests, and forwards the events to Kafka for downstream processing.

The service validates incoming webhook requests using SendGrid's event webhook signature verification, parses the JSON event data, and produces events to Kafka for asynchronous processing. Failed events are logged to AWS S3 for debugging and analysis.

## Getting Started

### Requirements
Minimum requirements and recommended tooling:

- Python: 3.13
- UV (astral `uv` tool) for environment and task running (see `pyproject.toml` `tool.uv` configuration)
- Docker and Docker Compose (used for CI targets)
- Make (for provided Makefile targets)

Developer tooling (used in CI / recommended):

- `pytest` (unit tests)
- `ruff` and `mypy` (linting / type checks)

### Installation
```bash
# Install uv (if not installed)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install project dependencies
make env
```

### Configuration
For local development copy the provided example environment file to `.env`:

```bash
cp .env.example .env
```

After copying, edit `.env` and fill in any required values (for example AWS settings, Kafka configuration, or SendGrid public keys) before running the service or tests.

## Testing

### Unit tests
Run the project's unit tests with:

```bash
make test
```

## Linting and formatting
Run the linter with:

```bash
make lint
```

Run the code formatter with:

```bash
make fmt
```

## Project Structure

- `app/` - Main application code
  - `handlers.py` - Handler implementation for processing SendGrid webhook events
  - `models.py` - Pydantic models for request validation and event parsing
  - `config.py` - Configuration management (AWS, Kafka, SendGrid settings)
  - `auth.py` - Request authentication and signature verification
  - `main.py` - Lambda handler entry point
  - `modules.py` - Dependency injection configuration
  - `exceptions.py` - Custom exception definitions
  - `connectors/` - External service connectors
    - `aws/s3.py` - AWS S3 client for storing failed events
- `tests/` - Test suite
  - `unit/` - Unit tests
    - `test_handler.py` - Handler tests
    - `test_main.py` - Lambda entry point tests
    - `test_models.py` - Model validation tests
    - `conftest.py` - Pytest fixtures and configuration
    - `utils.py` - Test utilities

## Event Processing

### Supported Events
The service handles all SendGrid event types:
- `processed` - Email has been processed by SendGrid
- `deferred` - Email delivery has been deferred
- `delivered` - Email has been successfully delivered
- `open` - Email has been opened by recipient
- `click` - Link in email has been clicked
- `bounce` - Email bounced (permanent or temporary)
- `dropped` - Email was dropped
- `spamreport` - Email was marked as spam

### Event Filtering
- Events with `send_type: "fan_reply_forwarding"` are filtered out and not processed (return 200 status without further processing)

## Running locally
This AWS Lambda is not intended to be run locally — it is designed to handle real requests based on email event webhooks from SendGrid. For local checks use the unit tests (`make test`) and the linter/formatter (`make lint`, `make fmt`).