# aws-step-functions-local

The goal of this project is to see what [aws step function downloadable version](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-docker.html) can do to make development easier. It also experiments with running other containerized services locally, notably s3.

The simple state machine defined in `steps.json` does the following...
1. Read image URL as input, check if data exists in fake s3
	1. If data does not exist in s3, download and store
1. Read data from s3 and print ascii verison of image to logs


### Requirements
The following things are necessary to setup before getting started.
* [jq](https://formulae.brew.sh/formula/jq) - tested with v1.6
* [awscli](https://formulae.brew.sh/formula/awscli) - tested with v2.1.34

## Start Containers and Setup Fake AWS Infrastructure

```
$ run.sh setup steps.json
```
1. Builds and starts local containers via `docker-compose`
1. Creates state machine defined in `steps.json` using awscli
1. Create s3 bucket named `test` using awscli
1. Prompt for keypress to shut containers down

## Execute State Machine
```
$ run.sh exec input1.json one
```
1. States execution using `input1.json` as input and `one` and execution ID
1. Polls for `ExecutionSuccess` or `ExecutionFailed` status
1. Reports details of execution
1. Outputs ascii result


### Issues

#### Problem

The step function container needs to be able to send requests to the appropriate lambda function containers, but aws expects that you're using their [serverless application model cli](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-lambda.html) to configure and run everything. This means we would need a large amount of configuration that is useless outside of setting up a local environment. We need something that can fake being the aws lambda service to forward requests between specific functions and the state machine.

#### Investigation
1. Create `nginx_proxy` configured as `nginx-proxy` in `docker-compose.yaml`
1. Set proxy to be aws lambda location in `aws-stepfunctions-local-credentials.txt`
1. Capture requests from state machine
1. Create forwarder in `nginx_proxy/nginx.conf` 
	* :warning: Forwarding works, but failure in a function results in the task being marked as a success and the error response being fed to the next step
1. Setup [localstack](https://github.com/localstack/localstack) for the containerized step function module 
	* :warning: The same issue occurs, the container is just wrapping the latest [aws local step function jar](https://github.com/localstack/localstack/blob/master/localstack/services/install.py#L37)
	* :question: Could use localstack's lambda container, but that would require registering lambda functions using zip files or images from ecr, which would make local development very slow and cumbersome and containerized lambdas are part of their pro package
1. Create `flask_proxy` configured as `flask-proxy` in `docker-compose.yaml`
1. Examine responses from functions and throw HTTP 500 on error
	* :tada: This works! Execution of the state machine stops if a function fails
	* :warning: There may be other strange behaviors being hidden by having to mock out error responses, we should talk to aws about this

#### Conclusion
This setup isn't ready for primetime yet. We should use is with caution and talk to aws about having the step function container better match real behaviors.