Forum Discussion
DataflowGen2 Slow Performance during design is HORRIBLE
Hi ToddChitt,
I apologize for any inconvenience. Here are the two documents that explain Trail vs Premium. Please review the links below for a better understanding:
Trial Capacity (F64)- Typically time-limited (e.g., 60 days).
- May have restrictions or limitations compared to a paid capacity.
- Designed for evaluation purposes.
Purchased F64 Capacity
- A fully paid, production-ready capacity.
- Includes service level agreements (SLAs) and comprehensive support.
- Ensures uninterrupted access as long as the subscription remains active.
While both options provide comparable compute power, please note that the trial version is temporary and may have certain limitations compared to the purchased capacity.
Please refer to the below links for better understanding:
Fabric trial capacity - Microsoft Fabric | Microsoft Learn
What is Power BI Premium? - Power BI | Microsoft LearnAlso, kindly refer the below link for more information:
Microsoft Fabric concepts - Microsoft Fabric | Microsoft Learn
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
Hi ToddChitt,
Thank you for reaching out to the Microsoft Fabric Community Forum.
After reviewing the details you provided, I have identified a few workarounds that may help resolve the issue. Please follow these steps:
- Free Trial capacities (F64) have limited resources, which may affect performance. High usage of the workspace (by multiple users or processes) can lead to performance issues. Navigate to Fabric Admin Portal > Capacity Settings > Metrics. If usage is high, try working during off-peak hours. For extensive use, consider upgrading to a paid capacity (F2 or higher).
Plan your capacity size - Microsoft Fabric | Microsoft Learn - Instead of loading the file directly from SharePoint, please download it and store it in a Lakehouse first. Then, reference it in the Dataflow. If the file needs to be in SharePoint, ensure it is stored in OneDrive for Business rather than a SharePoint site, as ODFB offers better integration. Additionally, avoid unnecessary transformations in the Power Query Editor while designing the Dataflow.
- Instead of connecting Dataflow Gen2 directly to the Warehouse, follow these steps: First, load data into a Lakehouse. Then, use a Notebook or Pipeline to transfer data from the Lakehouse to the Warehouse. If you must use Dataflow Gen2, try creating a new, empty Dataflow and connect it solely to the Warehouse to check if the issue lies with the Dataflow creation itself.
- To enable fast copy, navigate to the appropriate Fabric endpoint, access a premium workspace, and create a Dataflow Gen2. On the Home tab, select Options, then go to the Scale tab and check the “Allow use of fast copy connectors” checkbox.
Fast copy in Dataflows Gen2 - Microsoft Fabric | Microsoft Learn
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Best Regards.
- Are you saying that a Trial Capacity (at F64) is NOT the same as a purchased F64 Capacity?
- I tried downloading the file (360 KB) and then uploading into a Lakehouse in the same Workspace. I can get to it, but for the Dataflow to 'figure out' the contents and show me the available sheets and tables (of which there are less than 10 total), i.e. the [Content] step, it just hangs. Maybe, once it did finish reading it, simply showed an icon of the file in the space where data is usually displayed. I was not able to take any further actions.
- For one of the Destinations, I wanted to send it to a Warehouse. The wizard will typicaly show me a list of Warehouses in the same Workspace. It does not, it just hangs.
- For another DF, I was trying to load data from Dataflow Gen1. To expand the "Dataflows" node during initial design, it just spins.
Hi ToddChitt,
After reviewing the details you provided, I have identified alternative workarounds that may help resolve the issue. Kindly follow these steps:- Yes, that’s correct. A Trial Capacity (at F64) and a purchased F64 Capacity are not the same.
- Use a Notebook or Data Pipeline to upload data from the Lakehouse to the Warehouse instead of Dataflow Gen2. Convert Excel files to CSVs for faster uploads. If using Excel, keep it as a Delta table in the Lakehouse before referencing in Dataflow.
- Load data into a Lakehouse table or Notebook (SQL or PySpark) to write to the Warehouse and create a new Dataflow Gen2 to check if the issue is Dataflow-related. Check connectivity in the Fabric Admin Portal under Capacity Metrics for throttling or high usage.
- Store Dataflow Gen1 output in a Lakehouse table as an alternative to Dataflow Gen2. Use a Notebook to retrieve Dataflow Gen1 results and transfer them to a Lakehouse or Warehouse. If necessary, create a new, simple Gen2 Dataflow to see if the issue is related to Gen1 connectivity.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Best Regards.
