Gareth eagar

• May 05, 2026
• software-engineering

Data Engineering With AWS: Closing Thoughts on This Book Retelling Series

We made it. Twenty posts. Fourteen chapters. One very thorough book about building data pipelines on Amazon Web Services.

When I started this retelling series back in December, I said the goal was to walk you through “Data Engineering with AWS” by Gareth Eagar in plain language, one chapter at a time. No jargon walls. No assumed knowledge. Just the concepts, the tools, and how they fit together. I hope I delivered on that.

• Apr 28, 2026
• software-engineering

Data Engineering With AWS Chapter 14: Wrapping Up the Learning Journey

This is post 20 in my Data Engineering with AWS retelling series.

We made it. Thirteen chapters of pipelines, transforms, orchestration, security, querying, visualization, and machine learning. Chapter 14 is the final chapter. It does not introduce a new AWS service. Instead, it zooms out. Way out. It looks at how data engineering works in the real world, shows you case studies from Spotify and Netflix, and points you toward where the field is heading next.

• Apr 21, 2026
• software-engineering

Data Engineering With AWS Chapter 13: Enabling AI and Machine Learning

This is post 19 in my Data Engineering with AWS retelling series.

Throughout this book, we have been ingesting data, transforming data, storing data, querying data, and visualizing data. All of that is incredibly useful on its own. But there is a whole other level where data gets really powerful: when you use it to teach a machine to make predictions.

• Apr 14, 2026
• software-engineering

Data Engineering With AWS Chapter 12: Visualizing Data With Amazon QuickSight

This is post 18 in my Data Engineering with AWS retelling series.

We have spent eleven chapters ingesting data, transforming data, cataloging data, querying data. But here is a simple truth: nobody wants to stare at 10,000 rows in a spreadsheet. Our brains are not built for that. We process pictures way faster than text. A well-designed chart can tell you in two seconds what would take twenty minutes to figure out from raw numbers.

• Apr 07, 2026
• software-engineering

Data Engineering With AWS Chapter 11: Ad Hoc Queries With Amazon Athena

This is post 17 in my Data Engineering with AWS retelling series.

You have a data lake. Terabytes of files sitting in S3 across landing zones, clean zones, and transform zones. The data is there. But how do you actually ask it questions? You could spin up a database, load everything into it, and then query. But that defeats the purpose of having a data lake in the first place.

• Mar 31, 2026
• software-engineering

Data Engineering With AWS Chapter 10: Orchestrating the Data Pipeline

This is post 16 in my Data Engineering with AWS retelling series.

Up to this point in the book, we have been doing everything by hand. Click this button in the console. Run that Glue job manually. Trigger a crawler. Upload a file. It works fine for learning. But imagine doing that in production, every day, at 3 AM, across dozens of data sources. No thanks.

• Mar 24, 2026
• software-engineering

Data Engineering With AWS Chapter 9 Part 2: Bridging Data Lake and Data Warehouse

This is post 15 in my Data Engineering with AWS retelling series.

In Part 1, we looked at Redshift internals – clusters, slices, distribution styles, sort keys. All the pieces that make a data warehouse fast. But a warehouse sitting in isolation is not very useful. Data needs to flow in from your data lake, and sometimes it needs to flow back out. Part 2 of Chapter 9 covers that bridge between S3 and Redshift, including Redshift Spectrum, the COPY and UNLOAD commands, and a hands-on exercise that ties it all together.

• Mar 17, 2026
• software-engineering

Data Engineering With AWS Chapter 9 Part 1: Loading Data Into a Data Mart With Redshift

This is post 14 in my Data Engineering with AWS retelling series.

Your data lake is humming along. Data lands in S3, gets cleaned, transformed, cataloged. Athena lets you run SQL queries on it. So why would you need anything else?

• Mar 10, 2026
• software-engineering

Data Engineering With AWS Chapter 8: Who Actually Uses All This Data?

This is post 13 in my Data Engineering with AWS retelling series.

We have spent the last several chapters ingesting data, transforming data, optimizing data. Pipelines everywhere. But here is the question nobody asks often enough: who is actually going to use all of this?

• Mar 03, 2026
• software-engineering

Data Engineering With AWS Chapter 7 Part 2: Transforming Data - Optimization and Business Logic

This is post 12 in my Data Engineering with AWS retelling series.

In Part 1, we covered the generic data preparation transforms: converting to Parquet, partitioning, PII protection, and data cleansing. Those transforms work on individual datasets and do not need much business context. Now we get to the transforms that actually create business value. The ones that combine multiple datasets, add context, flatten structures, and produce the tables that analysts and dashboards consume.

• Feb 24, 2026
• software-engineering

Data Engineering With AWS Chapter 7 Part 1: Transforming Data - The Basics

This is post 11 in my Data Engineering with AWS retelling series.

You have data sitting in your data lake. Raw CSV files, JSON dumps, database extracts. It is all there, technically available, but trying to run analytics on it is painfully slow and expensive. This chapter is about fixing that. Transforming raw data into something optimized, clean, and ready for actual use.

• Feb 17, 2026
• software-engineering

Data Engineering With AWS Chapter 6 Part 2: Ingesting Streaming Data

This is post 10 in my Data Engineering with AWS retelling series.

Part 1 covered batch ingestion – pulling data from databases into S3 on a schedule. But not all data waits politely for a nightly load. IoT sensors, vehicle telemetry, live gameplay events, social media mentions – this data streams in continuously and often needs to be processed in near-real-time.

• Feb 10, 2026
• software-engineering

Data Engineering With AWS Chapter 6 Part 1: Ingesting Batch Data

This is post 9 in my Data Engineering with AWS retelling series.

You have your whiteboard architecture from Chapter 5. You know who your data consumers are and what they need. Now it is time to actually move data. Chapter 6 covers data ingestion – getting data from wherever it lives into your AWS data lake. This first part focuses on batch ingestion from databases and files. Part 2 covers streaming.

• Feb 03, 2026
• software-engineering

Data Engineering With AWS Chapter 5: Architecting Data Engineering Pipelines

This is post 8 in my Data Engineering with AWS retelling series.

You have learned about data engineering principles, data architectures, the AWS toolkit, and data governance. Now comes the part where it all comes together. Chapter 5 is about designing an actual data pipeline. Not writing code yet. Just thinking. Planning. Drawing on a whiteboard.

• Jan 27, 2026
• software-engineering

Data Engineering With AWS Chapter 4 Part 2: Data Governance in Practice

In Part 1, we covered the theory: what data security and governance mean, how catalogs prevent your lake from becoming a swamp, and the core AWS services for encryption and identity. Now it is time to put it into practice.

• Jan 20, 2026
• software-engineering

Data Engineering With AWS Chapter 4 Part 1: Data Cataloging and Security

You can have the fastest data pipeline on the planet. You can have the slickest dashboards, the fanciest machine learning models, the most optimized Parquet files. None of it matters if your data gets stolen, mishandled, or dumped into a lake that nobody can navigate.

• Jan 13, 2026
• software-engineering

Data Engineering With AWS Chapter 3 Part 2: The AWS Toolkit - Analytics and Processing

In Part 1 we covered how data gets into AWS. Now comes the good part: what do you actually do with it once it is there? This post covers the services for transforming raw data, orchestrating multi-step pipelines, and letting people query and visualize the results.

• Jan 06, 2026
• software-engineering

Data Engineering With AWS Chapter 3 Part 1: The AWS Toolkit - Storage and Databases

Chapter 3 is massive. It is basically a catalog of every AWS service a data engineer will touch, from getting data in to getting answers out. So I am splitting it into two posts. This first part covers how data gets into AWS – all the ingestion services, the streaming tools, and the physical devices AWS will literally ship to your door.

• Dec 30, 2025
• software-engineering

Data Engineering With AWS Chapter 2: Data Management Architectures for Analytics

Chapter 1 gave us the “who” and “why” of data engineering. Now it is time for the “where.” Where does all that data actually live? How do organizations store, organize, and serve billions of rows of information so that someone on the business side can pull up a dashboard and make a decision before lunch?

• Dec 23, 2025
• software-engineering

Data Engineering With AWS Chapter 1: What Even Is Data Engineering?

If someone told you twenty years ago that data would become more valuable than oil, you would have laughed. But here we are. The most valuable companies on the planet are not drilling for crude. They are collecting, processing, and squeezing insights out of massive piles of data. And behind every one of those companies, there is a team of data engineers making it all work.

• Dec 16, 2025
• software-engineering

Data Engineering With AWS: A Book Retelling Series for the Cloud-Curious

Every company today is drowning in data. Clicks, transactions, sensor readings, log files, social media posts. It just keeps coming. But raw data sitting in a pile is useless. The real magic happens when someone builds the pipes that move it, clean it, reshape it, and deliver it to the people who need it.