Data Engineering With AWS Chapter 10: Orchestrating the Data Pipeline
This is post 16 in my Data Engineering with AWS retelling series.
Up to this point in the book, we have been doing everything by hand. Click this button in the console. Run that Glue job manually. Trigger a crawler. Upload a file. It works fine for learning. But imagine doing that in production, every day, at 3 AM, across dozens of data sources. No thanks.