Forum Discussion
Helper IData Flow connected to Gen2 block blobs Refresh issue
I created a new Data Flow from BI Service Premium Per user.
The Data source is a Gen 2 folder that gets 1 new blob block daily.
I thought when I set a Scheduel Refresh in the Data Flow it would only load the Block or Blocks it has not already refreshed from.
It apears evean though the blocks are all aproximetly the same size the schedual refresh is getting longer and longer as it apears to reloading all the Blocks in the folder every day back to the Dataflow.
Is this how it works.
Thanks
3 Replies
- collinq
Super User
Hi nrowey ,
I think you are wanting to use Incremental Refresh. Whether dataflow or desktop/service refresh the schedule refresh refreshes the entire data set. This link should help https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-understand-optimize-refresh#:~:text=Incremental%20refresh%20enables%20large%20dataflows%20in%20Power%20BI,refresh%20only%20data%20that%20needs%20to%20be%20refreshed.
nroweyI did not think you could do an incremental refresh pulling data from a Gen 2 blog.
- v-easonf-msft
Community Support
Hi， nrowey
Please check if this blog is useful to you.
Parquet, ADLS Gen2, ETL, and Incremental Refresh in one Power BI Dataset
Best Regards,
Community Support Team _ Eason
