Forum Discussion
Incremental refresh taking same time as full refresh
I have set up Inc.Refresh for 2 days on createddate and it's taking the same time as full refresh.
To test I have just one table and selected all the columns from the table and in ssms I can see only the last 2 days' data getting refreshed. But the time is the same as full refresh.
can you please help/suggest what's going wrong?
Inc.Refresh:
= Table.SelectRows(SS***BI, each [CreatedDateTimeStamp] > RangeStart and [CreatedDateTimeStamp] <= RangeEnd)
data from DB(Azure sql server)
2024-05-01 23:56:32.703
2024-05-01 23:56:32.703
2024-05-01 23:56:32.703
DDL:
[CreatedDateTimeStamp] [datetime2](3) NULL,
TIA
Sha
3 Replies
From the same DB, I have set up inc.refresh on another table --> It's working fine.
Full refresh: 1Hr
Inc refresh: 6 mins.
not sure why one table inc.refres working fine and for another table no.
ETL on Probelm table: Updates and Inserts
ETL on good table: Only Inserts
Incremental setup on both tables is on createddate.
Hi ShaShaSad ,
If you want to overwrite changes, then this is not something that an incremental refresh can do. To mitigate this, it is your responsibility to determine if and when to refresh old partitions or perform a full refresh across all partitions. Also, verify query folding; model refresh queries may be unfolded, resulting in long refresh times and high query hybrid engine resource utilisation. This happens because a query is created for each partition in the model. If the query is not collapsed and the data is not filtered in the data source, the engine will attempt to filter the data.
Here are some links as well as a solution that will hopefully help you.
Solved: Incremental Refresh taking much more time - Microsoft Fabric Community
Troubleshoot incremental refresh and real-time data - Power BI | Microsoft Learn
If your Current Period does not refer to this, please clarify in a follow-up reply.
Best Regards,
Clara Gong
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Thank you very much for youe reply.
Yes, as you are mentioned we will perform full refresh to handle updates to older data.
With regarding to Inc.refresh, i can see the native query.
select [_].[Number],
[_].[CreatedDateTimeStamp]
from [SSO].[vw_SSBI] as [_]
where [_].[CreatedDateTimeStamp] >= convert(datetime2, '2024-04-01 00:00:00') and [_].[CreatedDateTimeStamp] <= convert(datetime2, '2024-04-02 00:00:00')anything i need to change?
If you see i'm getting just 2 columns. One is number and other one createddate used in inc.refresh setup.
just 2 columns taking so long!
