Monitoring Data Pipelines - Study Notes From Data Engineering With Python Ch 9
You built a data pipeline. It is idempotent, uses atomic transactions, and has version control. It is production ready. But can you tell when it breaks?
Elasticsearch
Working With Databases - Study Notes From Data Engineering With Python Ch 4
Most data pipelines start with a database. Most of them end with one too. Chapter 4 of Paul Crickard’s book is about connecting Python to databases and moving data between them. If the previous chapter was about flat files, this one is where things get real.
Building Your Data Engineering Setup - Study Notes From Data Engineering With Python Ch 2
Chapter 1 was all theory. Now it’s time to actually install stuff. Chapter 2 of Data Engineering with Python by Paul Crickard is a setup chapter. You install the tools, configure them, and make sure everything talks to each other.