Performance articles

Every BricksNotes article on Performance: 16 practical pieces on Databricks and data engineering, newest first.

Deletion Vectors in Delta Lake: Why Deleting One Row No Longer Rewrites Your Data
Spill (Memory) and Spill (Disk) in Apache Spark: What They Actually Mean
Your Databricks Job Took 42 Minutes Instead of 8. What Changed?
Every Data Engineer Faces These 50 Problems. The Best Engineers Know the Solutions.
The checkpoint trap is finally fixable: resize stateful Spark streams without starting over
You partitioned the table. It got slower. Partitioning and liquid clustering, explained.
The small files problem in Spark and Databricks, explained simply
The table had 40,000 tiny files. The small file problem in Databricks, explained
Same result, 3 seconds or 30 minutes. Query optimization in Databricks
The join that ran for seven hours. Broadcast and shuffle joins in Databricks
All BricksNotes articles