Technical deep dives, architecture patterns, and best practices in data engineering.
Also published on Medium · Newsletter
Most Databricks streaming failures don't look dramatic. No cluster termination, no red wall of errors. Just a job that says RUNNING while your customers report nonsense.
Learn how to inspect the Delta transaction log to understand your partition size distribution and make informed partitioning decisions.
Deep dive into Z-ordering, data skipping, and compaction strategies to maximize Delta Lake performance.
Proven strategies for optimizing Databricks cluster configurations and reducing cloud infrastructure costs.
Exploring medallion architecture, data mesh, and other patterns for building scalable lakehouse platforms.