Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Dataflows and large datasets

I would like to know what exactly the recommendation is for using dataflows with large data sources.   Currently, when I make changes to a dataset that is connected to a database, I need to restric...
Need Help
v-xicai's avatar
v-xicai
Community Support
6 years ago

Hi Anonymous ,

 

You can set incremental refresh for dataflow in Power BI Service via clicking incremental refresh icon and needn't to republish , see more detail: Using incremental refresh with Power BI dataflows .

You can also learn more: How to Use Dataflow to Make the Refresh of Power BI Solution FASTER!

 

Best Regards,

Amy

 

Community Support Team _ Amy

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Thanks v-xicai but setting the incremental load in itself is not the issue.  I have done this several times and I'm happy with it, and also with connecting dataflows to other sources etc.

 

The problem is figuring out the best way to work with existing datasets based on either source data (in databases) or from dataflows, that have quite a lot of history.

 

Please see my reply to Jeff for additional information.

 

The single question that I would like to be answered is:

"What is the optimal way to make changes to a large dataset, with signficant historical data, that has a minimal impact on my organisation's users?"

 

Thanks

  • spoutrain's avatar
    spoutrain
    Regular Visitor
    6 years ago

    Hi Anonymous ,

     

    Have you found a solution since then ? I face the same issue with multiple huge dataflows used in the dataset, which takes tremendeous amount of time to download, and then to upload back to PBI Service. It has not real value, since once the dataset is published back to PBI Service, it will refresh anyway 😕

     

    Thanks !