Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Incremental Refresh on a dataset with Dataflows as a data source

Hi,   I have a question about implementing Incremental Refresh on a dataset that connects solely to Dataflows. I tried setting it up but it appears Dataflows do not support query folding.   Few q...
Dataflow
Need Help
Anonymous's avatar
Anonymous
Not applicable
6 years ago

Hello Anonymous,

 

Any updates from your/Microsoft side ?

 

Could you please officially confirm, that our requirement

  1. Incremental Refresh of Dataflow
  2. Incremental Refresh of Datasets depending on this Dataflow

is technically possible and supported by Microsoft.

 

Thanks and best regards

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Hi GilbertQ ,

Sorry to bother you again. I read the related blogs you wrote about increment refresh, it have good explanation of increment refresh. May I ask for your help answer Anonymous 's question about increment refresh of dataflow? Thanks for your help in advance.

Any updates from your/Microsoft side ?

 

Could you please officially confirm, that our requirement

  1. Incremental Refresh of Dataflow
  2. Incremental Refresh of Datasets depending on this Dataflow

is technically possible and supported by Microsoft.

Best Regards

Rena

  • GilbertQ's avatar
    GilbertQ
    Icon for Super User rankSuper User
    6 years ago
    Hi there

    As far as I know it is possible to incrementally refresh the dataflow.


    I do not think it is currently supported in Power BI pro to incrementally refresh off a dataset depending on a dataflow.
    • Anonymous's avatar
      Anonymous
      Not applicable
      6 years ago

      Hi GilbertQ ,

      Thanks for your quick response and answer !

       

      According to Microsoft (https://docs.microsoft.com/en-us/power-bi/transform-model/service-dataflows-incremental-refresh) "Dataflow incremental refresh and dataset incremental refresh are designed to work in tandem", i.e. in an Enterprise Datawarehouse (DWH) setting the Dataflows (usually many of them) ingesting large amounts of data every night into Power BI Azure (implicit) datalake need to use the Incremental Data Refresh and the depending Datasets, that have to be refreshed after the ingest, must be incrementally refreshed as well to keep within the timelimits set by a nightly dataload (usually only a few hours after all other source systems have submitted the data to the DWH).

       

      If this (incremental refresh of Dataflows and incremental refresh of depending Datasets) is not supported/working then Power BI is not really ready for Enterprise DWH reporting/analyses.

       

      What is your opinion on this or do you see an alternative way to use Power BI in an Enterprise scenario ?

       

      Thanks and best regards

       

      • GilbertQ's avatar
        GilbertQ
        Icon for Super User rankSuper User
        6 years ago
        Hi there

        Always if you want an enterprise DW then you would need to build an enterprise data warehouse that would be separate to dataflows in my opinion.

        Whilst dataflows are great they do not have all the features that you would expect to have in a data warehouse table.