Forum Discussion
Query dependencies - avoid full refresh
Hi
Say I have a FACT table, which takes a long time to refresh. I also have a lookup table, which does not take a long time to refresh.
The very last step of the FACT table query is joining FACT and lookup - one column from lookup into FACT.
Question: Can I somehow make changes to the lookup table, and update the FACT table accordingly, without refreshing the entire FACT table again, which as previously mentioned, takes a very long time?
Thanks
/Henrik
No. Power Query has to process the entire dataset when it loads it into the model. The only way around this is using Incremental Refresh, and that is only availble in data sources that support it. Flatfiles, text, excel, SharePoint Lists definitely do not. Relational databases may. SQL Server does for example, as does PostGRE SQL, but MySQL does not. It depends on your dataset.
That said, Incremental Refresh is a service side thing. It won't help you when you are developing in Power BI Desktop. What you could do is temporarly create a filter on the big data table to limit to a few thousand records, then remove the filter when you are ready to look at the entire dataset and publish.
4 Replies
Not that I am aware. Perhaps if you implemented a measure.
Maybe... depends on what your data source kind is. E.g. is it SQL, Azure Tables, or GBQ? Something else?
No. Power Query has to process the entire dataset when it loads it into the model. The only way around this is using Incremental Refresh, and that is only availble in data sources that support it. Flatfiles, text, excel, SharePoint Lists definitely do not. Relational databases may. SQL Server does for example, as does PostGRE SQL, but MySQL does not. It depends on your dataset.
That said, Incremental Refresh is a service side thing. It won't help you when you are developing in Power BI Desktop. What you could do is temporarly create a filter on the big data table to limit to a few thousand records, then remove the filter when you are ready to look at the entire dataset and publish.
