March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early bird discount ends December 31.
Register NowBe one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now
Hello to the forum,
I set up a dataflow in a premium workspace. In it I created a folder query, which imports several Excel files from a folder. To improve query performance I now want to enable incremental refresh.
The YouTuber Hoosier BI describes in his video Icremental Refresh with SharePoint Files that the filtering with the two parameters "RangeStart" and "RangeEnd" must be done before the step in which the folder query uses the function to extract data from the Excel files. However, his video describes the procedure in Power BI Desktop and not in the data flow.
When I tried to apply this guide to my folder query in the dataflow, I got the following error:
Dataflow cannot be saved
Model is invalid because it contains duplicate query names. Error: This document contains a duplicate member "RangeStart".
I then simply omitted the two parameters and then set the incremental update. It works perfectly.
My questions are now:
1. So is it correct to omit the "RangeStart" and "RangeEnd" parameters in the incremental update in the data flow?
2. Does incremental updating actually make any sense at all? Yes, Hoosier BI says that the filtering must be done before the function that extracts the data from the Excel files. But without the two parameters, I can't specify when the filtering takes place. So my concern is that Power Query will import all the data from the folder and then apply the logic from the incremental update (which would make that pointless).
What do you think?
What I meant is if you managed to reduce the refresh time. After some testing, I successfully configured incremental refresh in dataflows as follows:
I don't know if this is the best way but it worked.
Greetings!
Hi @Jan_Trummel , when you ommited the parameters and set up the incremental refresh, did you see and improve in refresh time? Because when I do it the refresh time increased. I'm also using dataflow and my datasource is files in sharepoint.
Hello @hel , thank you for your answer!
What do you mean with "see and improve in refresh time"? How can I improve it?
Tank you and have a nice weekend!
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.
Arun Ulag shares exciting details about the Microsoft Fabric Conference 2025, which will be held in Las Vegas, NV.
User | Count |
---|---|
37 | |
22 | |
20 | |
10 | |
9 |
User | Count |
---|---|
60 | |
56 | |
22 | |
14 | |
12 |