Blog Post

Data Engineering Community Blog
2 MIN READ

πŸš€ From 60 Minutes to 30 Minutes: Optimizing Spark JSON Processing & Delta Lake Backfill

NHariGouthami's avatar
NHariGouthami
Microsoft Employee
11 months ago
Recently, I faced what looked like a β€œsmall” data engineering challenge: We added a new column (ClientPlatform) to our data pipeline. Sounds simple, right? But there was a catch β€” 30 days of histor...
Published 11 months ago
Version 1.0