Forum Discussion
Dataflow vs Dataset refresh
>So I assume the dataflow is much like a data storage component on its own that manages the updating from the data source, wherever that may be
That's correct, technically PBI's Dataflow uses Azure Data Lake Gen2 for storage.
One use case I plan to use this dual refresh structure for, is to handle sources (e.g. static files) that don't need to be refreshed in Dataflows where they'll be imported but not under scheduled refresh. I've found PBI's scheduled refreshes to fail easily, so cutting down the service's scheduled refreshes to sources that actually need to be refreshed should lower incidents (e.g. web API timeouts, credential issues etc.).
Hi,
After internal checking, it seems that the next refresh time update (without browser refresh) was fixed and is should be available in the following updates.
I will keep monitor it.
Thanks,
Assaf
Anonymous wrote: Ideally Dataset should work like a "Direct" Querying to Dataflow, not an Import
Let me try and explain why I disagree. That is really not "ideal", as Direct Query is meant to be used over structured data sources, not unstructured data lakes. Azure Data Lake is not supported as a source for DQ:
https://docs.microsoft.com/en-us/power-bi/desktop-directquery-data-sources
Moreover Import performs better than DQ and is really Power BI's preferred mode unless you have too much data or need real-time updates.
You can make the case that it should be easier to sync dataflow and dataset refreshes, but I think you're possibly confused about some of the architectural options offered by Power BI. The fact that you're using dataflows as a data source in a dataset should not limit how that dataset can work or be refreshed.
If I have a Power BI report with a single data source that is a Dataflow, and I schedule a refresh of the dataset in the service - that "refresh" is just pulling down the data already there in the dataset? i.e. the dataset refresh is not triggering another refresh in the lake?
thisisausername dataflow and dataset refreshes are completely separate. Refreshing one does not automatically refresh the other, if you want to trigger one refresh when the other is done you'll have to set that up using APIs.
Edit: watch the following video from Guy in a Cube if this still needs further clarification
I have dataflow currently running and dataset that link to the dataflow. However when i schedule an update for dataset,i realise that my dataflow also will be refresh if dataset being referesh. I'm not sure if i have wrongly set up my dataflow. How can i just update dataset without dataflow update concurrently?