Forum Discussion
Dataflow returns "old" data
Hi,
When I go into the dataflow editor, one table shows that it contains 30875 rows. However, when I refresh the table in the desktop app, the table has 31088 rows.
Can someone explain why does this happen?
This is not the first time I experienced this. Eventually, it always got the correct data but it is super annoying that I cannot work with the live data when I am using dataflows.
Please double check if you have the below setting enabled on your dataflow:
I am expecting you are using DirectQuery in dataflow as otherwise without refresh of data done at different times this issue can't happen.
Here is the link which might explain the functioning of direct query in Dataflows:
https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-premium-features
4 Replies
Dataflows read and store data from their source, and are refreshed (just like Datasets) on a regular basis.
What is probably happening here is that the Dataflow is refreshed, say at 10:00am, and stores 30875 rows. The datasource carries on growing and at 11:00am you connect to the datasource through Power BI Desktop (not the dataflow) and get 31088 rows. Later on the Dataflow could be refreshed at 12:00pm and "catches up" to get 31088 rows.
Take a look at the Dataflow settings and you will see the refresh schedule, that is when the Dataflow reads records from the source and stores them. One of the benefits of Dataflows is the fact that they store a copy of the data, ready for later usage, so they only hit the datasource at scheduled times so not slowing it down. You may want to connect to the Dataflow in Power BI desktop rather than connect directly to the datasource, to see the stored records rather than the live ones.
Hope this helps
Stuart
Thank you for your reply Burningsuit
I understand your point but that is not the case.
In Power BI Desktop I am connected to the dataflow, not directly to the datasource.
Refresh history of the dataflow:
In mssql the table returns 30.876 rows:
Dataflow editor also returns 30.876 rows:However, when I refresh the table in Power BI Desktop from the dataflow, it returns more rows.
Please double check if you have the below setting enabled on your dataflow:
I am expecting you are using DirectQuery in dataflow as otherwise without refresh of data done at different times this issue can't happen.
Here is the link which might explain the functioning of direct query in Dataflows:
https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-premium-features
I realized I overlooked something and this whole conversation is irrevelant.
Thank you for suggesting ideas I might have missed.
