Book: Data Pipelines with Apache Airflow
Authors: Bas Harenslak and Julian deRuiter
ISBN: 9781617296901
Azure gets the same treatment AWS got in the previous chapter. Map Airflow components to managed services, survey the provider package, then build the movie-ranking pipeline again. Same goal, different cloud.
Book: Data Pipelines with Apache Airflow
Authors: Bas Harenslak and Julian deRuiter
ISBN: 9781617296901
Chapter 15 is the bridge chapter. You spent the last few chapters building DAGs, custom operators, and a real project. Now the book asks a harder question: where does all of this actually run?
Chapter 12 is the one where everything moves to the cloud. If you’ve been following along, we’ve been talking about databases, pipelines, data quality, security, governance, and big data. All of that can run on your own hardware. But most teams today don’t do that. They use cloud providers. This chapter explains why, and more importantly, how.
Previous: Cloud Computing for Big Data: An Introduction
In the last post, we looked at the basic models of the cloud (IaaS, PaaS, and SaaS). Today, we’re talking about the “where” and the “who.” When you decide to move your big data to the cloud, you have to choose a deployment model and a provider.