Forum Discussion
Scott_Powell
Advocate IV
Advocate IVHow to choose optimal spark environment settings?
Hi all, I'm working on a notebook with Spark SQL to move and transform data between our bronze and silver lakehouses. In general, most of our ETL is joining multiple tables and writing an output tabl...
frithjof_v
Community Champion
Community ChampionThere is something called Autotune in preview.
I must be honest, I haven't tested it. I haven't even read about it. I just heard the name. Perhaps it is relevant for what you're asking for: https://learn.microsoft.com/en-us/fabric/data-engineering/autotune?tabs=sparksql
frithjof_vCommunity Champion
Community ChampionHere's a Fabric Espresso video on the Autotune feature:
https://youtu.be/iv1kAz_ShEo?si=arP4FA7wYhJisnvc
Unfortunately, I haven't had the time to watch it myself yet.