Chapter 6: Orchestrating Data Pipelines With AWS Step Functions
Book: Data Engineering with AWS: A Comprehensive Guide to Building Robust Data Pipelines
Author: Brian Paul
ISBN: Not listed in the source edition
Step functions
Data Engineering With AWS Chapter 10: Orchestrating the Data Pipeline
This is post 16 in my Data Engineering with AWS retelling series.
Up to this point in the book, we have been doing everything by hand. Click this button in the console. Run that Glue job manually. Trigger a crawler. Upload a file. It works fine for learning. But imagine doing that in production, every day, at 3 AM, across dozens of data sources. No thanks.