Book: Data Science: The Hard Parts
Author: Daniel Vaughan
ISBN: 978-1-098-14647-4
Publisher: O’Reilly, 2024
Previous: Narratives and Storytelling in Data Science | Next: Simulation and Bootstrapping
This is post 18 in my Data Engineering with AWS retelling series.
We have spent eleven chapters ingesting data, transforming data, cataloging data, querying data. But here is a simple truth: nobody wants to stare at 10,000 rows in a spreadsheet. Our brains are not built for that. We process pictures way faster than text. A well-designed chart can tell you in two seconds what would take twenty minutes to figure out from raw numbers.
Chapter 5 is where Boyan Angelov gets practical about the question everyone dances around: which language should you actually use for which job?
You ran the analysis. Got your numbers. Built a model. Now you need to show people what you found. And here is where most data people trip up. They pick the wrong chart, overload it with details, and the audience walks away confused.
You spend weeks building pipelines, modeling data, setting up orchestration. Everything works. Data lands in BigQuery clean and on time. And then someone from the business side asks: “So… where do I see the numbers?” That is exactly where Chapter 7 picks up. All that upstream work has to end somewhere useful, and for most organizations that somewhere is a dashboard.
Previous: Flink Connectors and Event Time: Mastering the Stream
You’ve done the hard work. You’ve set up a Hadoop cluster, written MapReduce jobs, and built real-time pipelines in Spark and Flink. You have “insights.” But here’s the problem: nobody wants to look at a raw HDFS file or a console log.