Forum Discussion
Power Query staging queries
In that case I would personally load the seperate views and use them as the staging step. If your datawarehouse also contains other views/tables I wouldn't load them into Powerquery/PowerBI. But this might also be personal preference.
For me the main principles would still remain.
- I would keep it simple and avoid loading data that doesn't end up in your report
I would recommend creating different staging dataflows per view. This makes the different dataflows less complex and easier to monitor. When you load everything seperatly you have the following advantages:
- Less complexity
- Easier to analyze performance of the different views/sources
- You can set different refresh schedules
With the help of power automate and a PowerBI streaming dataset you can create a monitoring report for your dataflows. When all staging tables are in one dataflow you only see the performance of that one single dataflow. When you have seperate dataflows you can monitor the performance of all the different dataflows.
Also take a look at https://learn.microsoft.com/en-us/power-query/dataflows/best-practices-developing-complex-dataflows
Microsoft also recommends to keep it simple. Don't put too much complex dataflows together for performance and complexity reasons.