Kibana

• Jun 19, 2026
• data-engineering

Building a 311 Data Pipeline With NiFi and Kibana

Book: Data Engineering with Python
Author: Paul Crickard
ISBN: 978-1-83921-418-9

Chapter 6 is where the book stops feeling like a tutorial and starts feeling like a real project. Crickard builds a pipeline that pulls citizen service requests from SeeClickFix (think 311-style complaints: graffiti, potholes, abandoned cars, needles, people ignoring social distancing during COVID). The data lands in Elasticsearch. Then you build a Kibana dashboard on top. He says he still runs this pipeline every 8 hours. That detail matters. This is not a throwaway demo.

• Mar 23, 2026
• Big Data

Real-Time Visualization With Elasticsearch and Kibana

Trino is great for querying your historical data on S3, but for real-time streams and text-heavy search, you need something different. In the second half of Chapter 9, Neylson Crepalde introduces the industry standard for real-time analytics: Elasticsearch and Kibana.

• Feb 21, 2026
• software-engineering

Monitoring Data Pipelines - Study Notes From Data Engineering With Python Ch 9

You built a data pipeline. It is idempotent, uses atomic transactions, and has version control. It is production ready. But can you tell when it breaks?