- Goal: answer some questions related to rolling out new asset pipeline for audio - What effect will changing timing of moving assets into AVL have on delivery times? - How many workers will we need for this to work? - Will we need to take another or additional approach? - Approach: run some simulations based on past year of data and see what we learn. 1. Gather data: - Did some querying in snowflake to make an approximation of when assets would move, if new system had been in place - When was earliest encoding order for a particular release? - How many files and how many bytes would be moved? - Gathered for 12 months, 11/2019 through 10/2020 - see - query.sql - data in data/ directory - sims/dac/input_data.py 2. Simulate: - Use simpy (https://simpy.readthedocs.io/) to simulate various combinations of parameters, notably worker count and bandwidth - Simulations recorded projected wait time (time for assets to be moved into local storage) for each release - Also did some process pooling and caching since can take a while - See - main.py - sims/dac/simulation.py 3. Visualize: - Use pandas (https://pandas.pydata.org/) for analysis, generating plots - See - sims/dac/plotting.py More to do: - Add noise for download speed - Validate bandwidth assumptions - Simulate outages - Investigate spikes