Forum Discussion
Loading data from ADLS Delta Table into Spark Dataframe is slow
Hi there, I'm experiencing slow read times when loading data from delta tables into data frames using PySpark in Synapse notebooks. This does not include the time taken for the Spark cluste...
Hi Anonymous
Where are you executing this query, is it in Fabric/Synapse. If it's in synapse what is the spark pool size used to run the notebook?
If you are using Fabric, what type of environment is it, Trail/Dedicated Capacity, if it's dedicated capacity what is the size of sku and node size, if it's trail what is the node size used?
In Synapse serverless, how did you test it, is it simply by select * from table or any other way?