Forum Discussion
Scott_Powell
Advocate IV
Advocate IVHow to choose optimal spark environment settings?
Hi all, I'm working on a notebook with Spark SQL to move and transform data between our bronze and silver lakehouses. In general, most of our ETL is joining multiple tables and writing an output tabl...
frithjof_v
Community Champion
Community ChampionThere is something called Autotune in preview.
I must be honest, I haven't tested it. I haven't even read about it. I just heard the name. Perhaps it is relevant for what you're asking for: https://learn.microsoft.com/en-us/fabric/data-engineering/autotune?tabs=sparksql
Scott_PowellAdvocate IV
Advocate IVhi frithjof_v , I had heard about the autotune back at the Fabric conference in Vegas, but didn't realize it was actually available yet, and that it was disabled by default. I'll definitely turn this on and see what happens, and post back.
Thanks for the heads up!
Scott