Forum Discussion
DP-700 Microsoft Learn Question
Hi tan_thiamhuat ,
I’d encourage you to submit your detailed feedback and ideas via Microsoft's official feedback channels, such as the Microsoft Fabric Ideas. Feedback submitted here is often reviewed by the product teams and can lead to meaningful improvement.
Thanks,
Prashanth Are
MS Fabric community support
Your company uses a lakehouse architecture with Microsoft Fabric for analytics. Data is ingested and transformed using Microsoft Data Factory pipelines and stored in Delta tables.
You need to enhance data transformation efficiency and reduce loading time.
What should you do?
(a) Configure Spark pool with more worker nodes. (wrong)
(b) Use session tags to reuse Spark sessions.
This answer is correct.
Using session tags to reuse existing Microsoft Spark sessions minimizes startup time and enhances the efficiency of data transformation processes. Configuring the Microsoft Spark pool to use more worker nodes might seem beneficial for handling more tasks but does not directly improve transformation efficiency.
--> I find the answer above not really correct: My version below:
Why (a) is correct:
(a) Configure Spark pool with more worker nodes.
More worker nodes = more compute resources.
This allows Spark to process partitions in parallel, reducing the overall time it takes to transform and load data.
Especially helpful for large datasets stored in Delta tables.
Why (b) is not optimal here:
(b) Use session tags to reuse Spark sessions.
Session reuse via tags helps reduce startup time for small or interactive workloads.
But it does not improve the actual data transformation performance or reduce load time significantly for heavy ETL workloads.
It is more about efficiency of session management, not compute scale.