# S3_bulk_decompress tool

## Usage(run script)

    decompress --help
    usage: decompress [-h] --bucket-archive BUCKET_ARCHIVE [--archive-keys ARCHIVE_KEYS] 
           --dsp-settings-path DSP_SETTINGS_PATH --bucket-decompressed BUCKET_DECOMPRESSED
           [--force] [--debug] [--json]

    required arguments:
      -h, --help            show this help message and exit
      --bucket-archive BUCKET_ARCHIVE
                            S3 bucket for source files
      
      --dsp-settings-path DSP_SETTINGS_PATH
                            S3 path to dsp settings
      --bucket-decompressed BUCKET_DECOMPRESSED
                            S3 bucket for decompressed files
      --archive-keys ARCHIVE_KEYS
                            Coma-separated s3 keys to source files
      
    optional arguments:
      --max-workers         pool of at most max_workers threads to execute calls asynchronously
      --force               flag to launch decompression in force mode, if set - 
                            files in destination bucket will be overridden during decompression even if 
                            they are exist.

      --debug               additional log level
      --json                json output

ATTENTION: If --archive-key  not set, all files. from source bucket will be decompressed

## Examples

     decompress --bucket-archive="stage-sme-data-archive" \
     --archive-keys="spotify/streams/v2/report_date=2020-02-24/report_licensor=theorchard/streams_20200224_AF.gz,spotify/streams/v2/report_date=2020-02-24/report_licensor=theorchard/streams_20200224_AE.gz" \
     --dsp-settings-path="s3://dev-delphi-configs/dsp-specific-settings.json" \
     --bucket-decompressed="dev-delphi-slz-download-jobs" --debug --force

## run with docker
    make docker/image/build
    docker run -it -v ~/.aws/:/app/.aws/ bulk_decompress_task:develop --debug --bucket-archive=stage-sme-data-archive --dsp-settings-path=s3://dev-delphi-configs/dsp-specific-settings.json --bucket-decompressed=dev-sme-data-decompressed --archive-keys=spotify/streams/v2/report_date=2020-02-24/report_licensor=theorchard/streams_20200224_AF.gz