# Visualise and analyze cache-related Snowflake performance issues

This script runs daily and populates the `CACHE_STATS_BY_QUERYID` table, which is basically returned by `SNOWFLAKE.INFORMATION_SCHEMA.QUERY_HISTORY_BY_WAREHOUSE()`, but enriched with `local_bytes` and `remote_bytes` columns. This allows to calculate percentage scanned from cache.

Also this script performs basic aggregations, creates and puts on S3 (currently to `dev-cucumbers/cache_stats/`) a graph of Snowflake caching related performance, and sends an event to Datadog.

![alt](2019-11-12_cache_stats.png)


### TBD: 
* Secretmanager integration
* Datadog integration
* `Correlation-Id` integration
* Deployment / scheduled execution