# Step Function and infrastructure

### 1. Create ECS TaskDefinition for downloader task

  Please follow the instruction in the `../task` folder `README.md`
  
### 2. Create Lambda function to read the config files

  - Create new lambda function in Python 3.6 using the `lambda-read-config.py` file
  - Assign a security role which allows reading from S3 bucket with configuration files

### 3. Create configuration settings files in S3

  - Put the `aws_config.json` and `task_config.json` in a S3 bucket of your choice
  - Update the link to the `aws_config.json` in `task_`conig.json` accordingly
  - In the `aws_config.json` update the parameters of ECS task created in step 1 (`Cluster ARN, Subnet ID, TaskDefinition ARN, Launch type`)
  
### 4. Create DynamoDB table

  - Create a DynamoDB table with the partion key: `job_id` (String)

### 5. Create Step Function state machine
  
  - Create new step function step machine using the `poc_state_machine_simple.json` file
  - Assign a security role to the step machine which allows to access S3, run ECS tasks and update DynamoDB table

### 6. Create and activate CloudWatch rule triggering the Step Function

  - Configure a new recurrent rule to trigger the Step Function created in previous step
  - Assign a security role which allows execution of the Step Function
  - Provide configuration files location in the rule input using the `task_config.json` file as an example
  - Activate the rule and examine the Step Function execution


### You may also examine the working PoC example in AWS

  - [Step Function](https://console.aws.amazon.com/states/home?region=us-east-1#/statemachines/view/arn:aws:states:us-east-1:475275892927:stateMachine:sf-poc-state-machine-simple)
  - [Lambda Function](https://console.aws.amazon.com/lambda/home?region=us-east-1#/functions/sf-lambda-read-config?newFunction=true&tab=graph)
  - [CloudWatch Rule](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#rules:name=sf_poc_cloudwatch_rule)
  - [ECS task definition](https://console.aws.amazon.com/ecs/home?region=us-east-1#/taskDefinitions/sfpoc-downloader/2)
  - [Configuration files in S3](https://s3.console.aws.amazon.com/s3/buckets/stepf-poc/config/?region=us-east-1&tab=overview)
  - [DynamoDB Table](https://console.aws.amazon.com/dynamodb/home?region=us-east-1#tables:selected=sf_poc_job;tab=overview)