# ETL Pipelines

This folder contains configuration for ETL pipeline exec jobs.

The difference of pipeline job from the regular exec job used in [flows](../flows) directories
is that the pipeline job combines several exec jobs.

## Existing Data Team pipeline jobs
* https://scheduler.theorchard.io/job/chartmetric-ingestion-pipeline/ - not migrated
* https://scheduler.theorchard.io/job/dbt-scheduler-analytics-pipeline/ - not migrated
* https://scheduler.theorchard.io/job/dbt-scheduler-analytics-pipeline-playlists-hourly/ - not migrated
* https://scheduler.theorchard.io/job/dbt-scheduler-analytics-daily-legacy-video-playlists-pipeline/ - not migrated

## How to create/migrate pipeline
* Follow instruction from [notion page](https://www.notion.so/Migration-SWF-Scheduler-configs-to-Jenkinsfile-8e6f76b1da91499c9a13ad04d003beff).
* Use [PR](https://github.com/theorchard/jenkins-config/pull/176/files) and [Jenkinsfile](sme-labelcopy-pipeline/Jenkinsfile) as reference for [swf-sme-labelcopy-pipeline](https://scheduler.theorchard.io/job/swf-sme-labelcopy-pipeline/)