# AWS Step Functions Management Tools

This repository contains two powerful tools for managing AWS Step Functions executions:

1. **Failed Executions Lister** (`program_list_failed_executions.py`) - Lists and analyzes failed executions
2. **Execution Retry Tool** (`bulk_start_new_execution.py`) - Creates retry executions from a list of execution names

## Features

### Failed Executions Lister
- 🔍 Lists failed executions for any Step Functions state machine
- 🎯 Takes state machine name as input (no need for ARNs)
- 📊 Shows detailed execution information including errors and causes
- � Filters by time range (default: past 5 days)
- �🔐 Supports multiple AWS credential methods
- ✅ Comprehensive error handling and user-friendly output

### Execution Retry Tool
- 🔄 Creates retry executions from a list of specific execution names
- 📝 Supports both command-line input and file input
- 🔢 Automatically increments retry count (e.g., ProcessData-001-retry-0, ProcessData-001-retry-1)
- 🧠 Intelligently handles existing retry executions (e.g., ProcessData-001-retry-2 → ProcessData-001-retry-3)
- 🔍 Dry-run mode to preview what would be created
- 🛡️ Safety confirmations for live execution mode
- 📋 Uses original execution input automatically
- 🎯 Perfect for targeted retries of specific executions

## Prerequisites

- Python 3.6 or higher
- AWS account with Step Functions access
- Appropriate IAM permissions

## Required IAM Permissions

Your AWS credentials need the following permissions:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "states:ListStateMachines",
                "states:ListExecutions",
                "states:DescribeExecution",
                "states:StartExecution",
                "sts:GetCallerIdentity"
            ],
            "Resource": "*"
        }
    ]
}
```



## Installation

1. **Clone or download the files**

2. **Create and activate a virtual environment:**
   ```bash
   # Create virtual environment
   python3 -m venv venv
   
   # Activate virtual environment
   source venv/bin/activate  # On macOS/Linux
   ```

3. **Install Python dependencies:**
   ```bash
   pip install -r requirements.txt
   ```

4. **Set up AWS credentials:**
   ```bash
   AWS_ACCESS_KEY_ID=
   AWS_SECRET_ACCESS_KEY=
   AWS_DEFAULT_REGION=
   AWS_SESSION_TOKEN=
   ```

## Usage

### Failed Executions Lister

#### Basic Usage
```bash
python program_list_failed_executions.py <state_machine_name>
```

#### Advanced Usage
```bash
# List failed executions from the past 7 days
python program_list_failed_executions.py MyStateMachine --days 7

# List failed executions from the past 1 day with max 50 results
python program_list_failed_executions.py MyStateMachine --days 1 --max-results 50

# Show help
python program_list_failed_executions.py --help
```

#### Command Line Options
- `state_machine_name`: Name of the Step Functions state machine (required)
- `--days`, `-d`: Number of days back to search (default: 5)
- `--max-results`, `-m`: Maximum number of results to return (default: 100)

### Execution Retry Tool

#### Basic Usage
```bash
python bulk_start_new_execution.py <state_machine_name> --executions <execution1> <execution2> ...
python bulk_start_new_execution.py <state_machine_name> --file <executions_file>
```

#### Advanced Usage
```bash
# Command-line input with dry run (recommended first step)
python bulk_start_new_execution.py MyStateMachine --executions ProcessData-001 ProcessData-002 --dry-run

# File input with dry run
python bulk_start_new_execution.py MyStateMachine --file executions.txt --dry-run

# Create retry executions from file
python bulk_start_new_execution.py MyStateMachine --file executions.txt

# Create retry executions from command line
python bulk_start_new_execution.py MyStateMachine --executions ProcessData-001 ProcessData-002

# Show help
python bulk_start_new_execution.py --help
```

#### Command Line Options
- `state_machine_name`: Name of the Step Functions state machine (required)
- `--executions`, `-e`: List of execution names to retry (mutually exclusive with --file)
- `--file`, `-f`: File containing execution names, one per line (mutually exclusive with --executions)
- `--dry-run`: Preview mode - shows what would be created without starting executions

#### File Format
```
# Comments starting with # are ignored
ProcessData-001
ProcessData-002
DataValidation-003
```

### Examples

#### Listing Failed Executions
```bash
# Default: Show failed executions from past 5 days
python program_list_failed_executions.py MyProcessingWorkflow

# Show failed executions from past 7 days  
python program_list_failed_executions.py DataPipeline --days 7

# Show only today's failures
python program_list_failed_executions.py DataPipeline --days 1

# Limit results to 25 executions from past 3 days
python program_list_failed_executions.py DataPipeline --days 3 --max-results 25
```

#### Creating Retry Executions
```bash
# First, always do a dry run to see what would be created
python bulk_start_new_execution.py MyProcessingWorkflow --executions ProcessData-001 ProcessData-002 --dry-run

# Create retry executions from command line
python bulk_start_new_execution.py MyProcessingWorkflow --executions ProcessData-001 ProcessData-002

# Create retry executions from file
python bulk_start_new_execution.py MyProcessingWorkflow --file sample_executions.txt --dry-run
python bulk_start_new_execution.py MyProcessingWorkflow --file sample_executions.txt

# Single execution retry
python bulk_start_new_execution.py MyProcessingWorkflow --executions DataValidation-2024-10-15-001
```

## Recommended Workflow

Here's the recommended workflow for using these tools together:

### 1. Investigate Failed Executions
```bash
# First, see what failed executions exist
python program_list_failed_executions.py MyStateMachine --days 7
```

### 2. Create Retry Executions (Dry Run)
```bash
# Always do a dry run first to see what would be created
python bulk_start_new_execution.py MyStateMachine --executions ProcessData-001 ProcessData-002 --dry-run
```

### 3. Execute Retry Creation
```bash
# After reviewing the dry run results, create the retry executions
python bulk_start_new_execution.py MyStateMachine --executions ProcessData-001 ProcessData-002
```

### 4. Monitor Results
- Check the AWS Step Functions console for new executions
- Monitor CloudWatch logs for any issues
- Use the lister tool again to verify the retries completed successfully

### Best Practices

1. **Always start with a dry run** to preview what will be created
2. **Use appropriate time windows** when listing failed executions
3. **Limit batch sizes** for large numbers of failures
4. **Monitor after creating retries** - ensure the retry executions complete successfully

## How Retry Naming Works

The Execution Retry Tool intelligently handles both original executions and existing retry executions:

### Original Executions
```
Input: ProcessData-001
Output: ProcessData-001-retry-0 (first retry)

Input: DataValidation-2024-10-15
Output: DataValidation-2024-10-15-retry-0 (first retry)
```

### Existing Retry Executions  
```
Input: ProcessData-001-retry-2 (already a retry)
Output: ProcessData-001-retry-3 (next retry in sequence)

Input: DataValidation-2024-10-15-retry-0 (already a retry)
Output: DataValidation-2024-10-15-retry-1 (next retry in sequence)
```

### Smart Base Name Detection
- The script automatically detects if an execution name is already a retry
- Extracts the base name (e.g., `ProcessData-001` from `ProcessData-001-retry-2`)
- Scans for all existing retries of that base name
- Creates the next retry in the sequence

This means you can retry any execution (original or retry) and get consistent, incrementing retry names.

## Troubleshooting

### Common Issues

1. **"No AWS credentials found" error:**
   - Ensure you've set up AWS credentials using one of the methods above
   - Run `aws sts get-caller-identity` to test your credentials

2. **"State machine not found" error:**
   - Check that the state machine name is correct (case-sensitive)
   - Ensure you're in the correct AWS region
   - Verify you have permission to list state machines

3. **"Access denied" errors:**
   - Check that your AWS credentials have the required IAM permissions
   - Ensure you're in the correct AWS account

4. **No failed executions shown:**
   - The state machine might not have any failed executions
   - Failed executions are only retained for 90 days in AWS

### Getting Help

- Check AWS CloudWatch logs for detailed error information
- Review AWS Step Functions documentation
- Verify IAM permissions are correctly configured

## Security Best Practices

- ⚠️ Never commit AWS credentials to version control
- 🔒 Use IAM roles when possible instead of access keys
- 🔄 Rotate access keys regularly
- 📝 Follow the principle of least privilege for IAM permissions

## Disclaimer

This project is AI-generated by Github Copilot - Claude Sonnet 5, and provided as-is for educational and operational purposes.