If you’re looking for the heavyweight champion of classification models, you’ve probably heard of Support Vector Machines (SVM). In Chapter 12 of Data Analytics for Finance Using Python, we see why this model is a favorite for high-precision tasks.
Chapter 11 is about leadership. But not the kind you see on LinkedIn where someone posts a sunset photo and writes “leaders eat last.” Diana is talking about something very different. Systems leadership is about improving how knowledge flows through your organization. Not about your title, not about your authority, not about how many people report to you.
Previous: Real-Time Analytics with Spark Streaming
In the last post, we looked at DStreams, the original way to do streaming in Spark. But things move fast in the tech world. Spark 2.0 introduced Structured Streaming, a new way to handle real-time data that makes things even simpler and more reliable.
If you’ve ever heard someone say a stock’s price is “three standard deviations away from the mean,” they’re talking about Z-Scores. In Chapter 11 of Data Analytics for Finance Using Python, we explore how to use this tool to find the “weird” data points that might actually be opportunities.
This is Part 2 of Chapter 10 from “Learning Systems Thinking” by Diana Montalion. Part 1 covered what modeling is and different modeling approaches. Now we get into the practical stuff. How do you actually use modeling to solve real problems? Diana brings back the MAGO case study to show us.
Previous: Spark SQL and Aggregations: Joining Your Data at Scale
Up until now, we’ve mostly talked about batch processing - looking at data that’s already sitting in HDFS. But what if you need to know what’s happening right now? What if you’re tracking a stock price, monitoring a server for hacks, or following a trending hashtag on Twitter? That’s where Spark Streaming comes in.
We just talked about the F-Test for comparing risk (variance), but what if you want to know if two stocks actually earn different amounts on average? In Chapter 10 of Data Analytics for Finance Using Python, we look at the T-Test.
Chapter 10 is a big one, so I’m splitting it into two parts. This is Part 1 of 2.
Diana opens with a Donella Meadows quote that sets the tone for everything that follows: get your model out where people can see it, invite others to challenge it. That’s the whole chapter in one sentence, really. But of course there’s much more to unpack.
Previous: Batch Analytics with Apache Spark: Faster Than MapReduce
In the last post, we looked at why Spark is so fast. Today, we’re getting into the nitty-gritty of how to actually use it. If you’re a SQL fan, you’re going to love this. Chapter 6 of Sridhar Alla’s book spends a lot of time on Spark SQL, and for good reason - it’s where most of the work happens.
We all see patterns. You get sick and you think back. Was I stressed? Did I sleep enough? Was someone around me sick? You compare symptoms to last time. You adjust your schedule.