Spark in Notebook taking more time to process the data.

Hi all,   I'm working on a diagnostic log ingestion engine built with PySpark and Delta Lake on Microsoft Fabric. My setup parses incoming ZIP logs from a server, transforms signal data per file in...