Forum Discussion
Power BI dataflow very slow
- 4 years ago
Hi georgec96,
Another option to consider, if you need all that data, is to use direct query for dataflows.
https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-directquery
I've had to use this recently for a large dataset and it works well. It then gives you the option to build aggregation tables in your model to improve performance but also keep the detailed data when necessary.
Caveat, this will only work with premium or premium per user as you need to be able to turn on the 'Enhanced compute engine settings' for the Dataflow.
Hope this helps.
Hi georgec96
A few things that can help:
- If you can, reduce the number of records you are using to undertake transformation / cleansing processes. If you can reduce it to say 20,000 records, you will find that the process to undertake the transformations will be far quicker. Once you have successfully completed these, then you can add the population of records for the load.
- Power BI takes a lot more time processing text-based fields than numerical fields. Two things to consider:
- Although 15 columns isn't much, remove what you don't need.
- If you have records with numerical values sitting in Text-based format, convert them to numerical values.
- The other thing to consider is the nature of your data. If your historical data remains that way and does not require retrospective change (e.g. you're not dealing with something like insurance claim data that the status may not be closed until 5-6 months from now, etc.), you might use parameters to limit the volume of data you apply in your refresh (i.e. if you have 2 years worth of data and only the last month is subject to change, you might create a parameter to only refresh the last month rather than the last 2 years).
Check out this link as well: https://docs.microsoft.com/en-us/power-bi/guidance/power-bi-optimization
There are a lot of ways you can reduce the load times and refresh rates. Hopefully the above will help guide you in the right direction.
All the best.
Theo