## Usage(run script)

    1. gen --help
    usage: gen [-h] --input INPUT [--debug]

	--yaml-dir-path INPUT		  Abs Path to dir with yaml files
	--config-dir-path INPUT		  Abs Path to dir with config file
	--config-name INPUT		  config file name(without extension) or "all" if you want to generate all configs at one time
	--environment -e INPUT		  env

    optional arguments:
    -h, --help            show this help message and exit
    --debug               additional log level

    2. check --help
    usage: check [-h] --input INPUT [--debug]

	--config-path INPUT		  Abs Path to config file
	--schema-path INPUT		  Abs Path to validation schema file


    optional arguments:
    -h, --help            show this help message and exit
    --debug               additional log level

## Examples

    1. gen --yaml-dir-path "/SONY/delphi-config/files" \
    --config-dir-path "/SONY/delphi-config/configs/dev" --config-name "dsp_config" \
    --environment "dev" --debug

    2. check  --schema-path "/SONY/delphi-config/files/dsp_config/schema.json" \
    --config-path "/SONY/delphi-config/dsp_config.json" \
    --debug


## Make commands

- `make venv` - create local virtual env(base.pip dependencies)
- `make testenv` - create local virtual env(test.pip dependencies)
- `make devenv` - create local virtual env(dev.pip dependencies)
- `make test` - run tests
- `make lint` - run pylint check
- `make build` - build package
- `make upload` - upload package on PyPI
- `make clean` - delete unnecessary files
- `make gen/all` - generate set of configs for all envs
- `make gen/<env>` - generate set of configs for a specified environment

- `make docker/login`  - login to AWS docker, aws configuration need to be set first
- `make docker/test/image` - pull image used for tests execution
- `make docker/test` - run tests in docker
- `make docker/lint` - run pylint check in docker
- `make docker/build` - build package in docker
- `make docker/upload` - upload package on PyPI from docker

- `make docker/image/build` - build Docker image
- `make docker/image/push` - push an image to a registry
- `make docker/image/pull` - pull an image from a registry

- `make docker/gen/all` - generate set of configs for all envs inside of a docker container
- `make docker/gen/<env>` - generate set of configs for a specified environment inside of a docker container

(when run `make test`, `make lint`  command, all dependencies from requirements/test will be installed.
When run `make venv` base.pip dependencies will be installed)

### configure AWS:
- to check if aws configuration was already set: `aws configure list`
- to configure `https://docs.aws.amazon.com/en_us/cli/latest/userguide/cli-chap-configure.html`

## HOW TO ADD A NEW CONFIG:
1.  To add a new config, structure of files must be:

        files/     - folder for all initial config files
            <name_of_config>/   - this folder should be named exactly as desired result name of config
                params/
                    dev.yaml
                    qa.yaml
                    stage.yaml
                    prod.yaml
                templates/
                    main.yaml
                    ...   - here should be all templates in .yaml with initial data
                schema.json

    flow of making configs is:

        params_from_params/ folder per environement applying to ->
            all templates from templates/ folder except main.yaml, convert this to iterable ->
                apply iterable to main.yaml ->
                    validate result with schema.json

2. If your config is not environment specific, you should not create folder params/ so your file structure will be:

        files/
            <name_of_config>/
                templates/
                    main.yaml
                    ...
                schema.json

3. Add name of your config to [files/config_names.yaml](files/config_names.yaml) 
   with desired output extension(json/yml).

## Config yaml structure
Description of params see [here](https://github.com/filtr/delphi-config#slz-configs)
#### dsp_config.json yaml structure
    dsp: vevo
    extension: tsv
    uow_active_hours_threshold: 36
    licensors:
      - licensor: sme
        types:
          - type: sony_standard_sales
            [OPT]subtype: Summary
            [OPT]schedule: '*/5 * * * *' - overrides schedule, set on dsp level
            [OPT]frequency: '0 0 * * 7'
            [OPT]uow_active_days_limit: 5 - overrides value from `dsp-specific-settings.json`
            [OPT]validation_thresholds: - overrides value from `dsp-specific-settings.json`
              low: 30
              high: 90
            [OPT]uow_complete_after_hours: 24
            versions:
              - valid_from: '2019-05-20'
                [OPT]valid_until: '2020-05-20'
                version: v1
              - valid_from: '2020-05-21'
                version: v2
    schedule: '*/10 * * * *'

#### dsp-complete-criteria.json yaml structure

    - complete_criteria:
      - context: content_owner_playlist_combined_a1
        [OPT]job_id: 397a715d-bfdf-4e52-aad3-328647ae116a
        [OPT]query: gras-queries/dim_participant.sql
        required: true
      licensor: sme
      partner: youtubereporting
      report_type: playlist_combined

#### dsp-specific-settings.json yaml structure
    apple:
      uow_active_days_limit: 14
      implementations:
      - rule: "*/*"
        label: apple
      days_offset: 1
      validation_thresholds:
        low: 40
        high: 50