# Architecture

This document describes how the code in this repository is organized and the
rules that keep it that way. It is the source of truth for "where does new code
go?".

## Goal

Keep responsibilities explicit and dependencies one-directional. A change to
business logic should never require changing a Snowflake client or a CLI
entrypoint, and vice versa.

## Layers

The `tadas/` package is split into five layers. A higher layer may import from
any lower layer; a lower layer must **not** import from a higher layer. Layers
on the same line are siblings — they must not import from each other.

```
tadas.cli              (top)
  └─▶ tadas.pipeline
        └─▶ tadas.monitoring  │  tadas.snowflake      (siblings)
              └─▶ tadas.domain
                    └─▶ tadas.platform                 (bottom)
```

These rules are enforced in CI by [import-linter] (`make check_imports`,
configured in [`.importlinter`](.importlinter)). Violations fail the build.

[import-linter]: https://import-linter.readthedocs.io/

### `tadas.cli` — entry points

Thin argparse + bootstrap. Sets up logging, Sentry, context, then calls a
single pipeline function. No business logic, no SQL, no pandas.

| Module | Entrypoint | Invoked by |
|---|---|---|
| `cli.launcher` | `python -m tadas.cli.launcher` | Docker (production `ENTRYPOINT`), local `make` |
| `cli.model` | `python -m tadas.cli.model {inference,sync-to-snowflake,delete-tables}` | `tadas/models/Makefile` |
| `cli.monitor_freshness` | `python -m tadas.cli.monitor_freshness` | `tadas/models/Makefile` `monitor` target |
| `cli.monitor_availability` | `python -m tadas.cli.monitor_availability` | same |
| `cli._argparse` | shared argparse helpers | other `cli.*` modules |

### `tadas.pipeline` — use cases

End-to-end orchestrations. One module per use case. This is the only layer
that mixes domain logic with I/O. Public surface is a single `run(...)` or
named function per module.

| Module | Public surface |
|---|---|
| `pipeline.inference` | `inference(report, report_date, model_config)` |
| `pipeline.publish` | `publish_to_snowflake(report, model_config)` |
| `pipeline.delete_tables` | `delete_report_tables(report, model_config)` |
| `pipeline.date_selection` | `find_latest_available_date(dsps)`, `find_common_max_value(lists)` |
| `pipeline.monitor_freshness` | `run()` |
| `pipeline.monitor_availability` | `run()` |

### `tadas.monitoring` — sensors (sibling of `tadas.snowflake`)

Pure monitoring concepts: sensor abstractions, freshness rules, threshold
helpers. Reads config, never touches the warehouse directly.

- `monitoring.sensors` — `Sensor`, `FreshnessSensor`, `CheckResult`,
  `run_freshness_check(...)`, `freshness_threshold()`.

### `tadas.snowflake` — warehouse adapter (sibling of `tadas.monitoring`)

Everything that talks to Snowflake. This layer owns the table-naming
convention, the SQL templates, and the connector wrapper.

| Module | Responsibility |
|---|---|
| `snowflake.client` | Connection, `query_snowflake_to_df`, `saveto_snowflake`, `execute`, etc. |
| `snowflake.tables` | `get_table_name(table_type, model_version, report)`, `TABLE_TYPES`, `TADAS_HISTORICAL_TABLE` — single source of truth for table names |
| `snowflake.facts_source` | Reads from DSP fact tables (data availability, tracks selection) |
| `snowflake.days_trending_generator` | SQL pull + `save_days_trending` wrapper around the pure algorithm in `domain.trending` |
| `snowflake.snowflake_publish` | Copies model output to final tables, syncs historical, saves combined DF |
| `snowflake.model_registry` | Calls Snowflake ML registry `model_predict_proba` |

### `tadas.domain` — pure business logic

No I/O. No `os.environ`. No `import snowflake`. Pure functions over
DataFrames and pure data definitions. This is the testable core.

| Module | Contents |
|---|---|
| `domain.constants` | `geos`, `REPORTS`, `REPORT_FINAL_TABLES`, `MODEL_VERSION_RE` |
| `domain.features` | `apply_adj_to_dod`, `add_is_any`, `combine_data_sources` |
| `domain.trending` | `prep_timeseries`, `calculate_consecutive`, `iron_out_trends` |
| `domain.trending_flags` | `apply_trending_flag`, `apply_trending_flags` |

### `tadas.platform` — cross-cutting infrastructure

May not import from any other `tadas.*` subpackage. Stdlib + 3rd-party only.

| Module | Responsibility |
|---|---|
| `platform.config` | Env-driven settings with typed `get(name)` |
| `platform.logging` | `init_logging`, `set_context`, `get_file_logger`, `redirect_print_to_logger` |
| `platform.sentry` | `init_sentry`, `set_context` |
| `platform.context` | Run-context JSON store (used by metrics) + `get_feed_name`, `get_context_id` |
| `platform.metrics` | `Metrics`, `DurationMetrics` |
| `platform.caching` | `cached`, `cached_df` (filesystem cache for DataFrames) |
| `platform.locking` | `try_lock`, `LockAcquireException` |
| `platform.dynamodb` | `set_overall_status`, `update_dynamodb_status` (feed status table) |
| `platform.jenkins` | `trigger_dbt_metrics` and Jenkins client |

## Outside the layered contract

### `tadas/models/`

Per-version model parameter sets (`t26XX_*/model_config.py`). Treated as a
versioned registry, not as application code. Each `model_config.py` exposes
constants (e.g. `MODEL_VERSION`, `REQUIRED_DSPS`, `ML_MODEL_FEATURES`,
`DBT_MODELS_COLUMNS`, `TRENDING_FLAGS`). Read by `pipeline.inference` and
`pipeline.publish` via `importlib`.

These configs must not import from `tadas.snowflake`, `tadas.pipeline`, or
`tadas.platform` — only stdlib and `tadas.domain.constants` if needed.

## CLI / Makefile entry surface

| Command | Module |
|---|---|
| `python -m tadas.cli.launcher` | top-level orchestrator (runs `tadas/models/Makefile`) |
| `python -m tadas.cli.model inference --report=…` | one inference run |
| `python -m tadas.cli.model sync-to-snowflake --report=…` | publish |
| `python -m tadas.cli.model delete-tables --report=…` | cleanup |
| `python -m tadas.cli.monitor_freshness` | freshness sensor against final tables |
| `python -m tadas.cli.monitor_availability` | freshness sensor per DSP feed |

`tadas/models/Makefile` wraps these into `inference`, `publish`, `clean`,
`monitor` targets that take `MODEL_VERSION` as an env var.

## Test layout

`tests/unit/` mirrors `tadas/`. A test for `tadas/domain/features.py` lives at
`tests/unit/tadas/domain/test_features.py`.

`tests/integration/` exercises against real Snowflake and is run separately
(not part of `make test`).

## Adding new code

Decision tree:

1. **Pure pandas / pure rule?** → `tadas.domain`
2. **Reads / writes Snowflake?** → `tadas.snowflake`
3. **Sensor / monitoring rule?** → `tadas.monitoring`
4. **End-to-end use case combining the above?** → `tadas.pipeline`
5. **argparse entrypoint?** → `tadas.cli`
6. **Cross-cutting infra (logging, config, metrics, …)?** → `tadas.platform`

If `make check_imports` rejects your change, the layering rule has been
violated. Fix it by moving the code rather than by widening the contract.