Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
ShaShaSad
New Member

Incremental refresh taking same time as full refresh

Hello,

 

I have set up Inc.Refresh for 2 days on createddate and it's taking the same time as full refresh. 

 

To test I have just one table and selected all the columns from the table and in ssms I can see only the last 2 days' data getting refreshed. But the time is the same as full refresh.

 

can you please help/suggest what's going wrong?

 

 

Inc.Refresh:

= Table.SelectRows(SS***BI, each [CreatedDateTimeStamp] > RangeStart and [CreatedDateTimeStamp] <= RangeEnd)

 

ShaShaSad_0-1714580105649.png

 

 

ShaShaSad_1-1714580105520.png

 

 

 

 

data from DB(Azure sql server)

2024-05-01 23:56:32.703
2024-05-01 23:56:32.703
2024-05-01 23:56:32.703

 

DDL:

[CreatedDateTimeStamp] [datetime2](3) NULL,

 

 

TIA

Sha

3 REPLIES 3
v-kaiyue-msft
Community Support
Community Support

Hi @ShaShaSad ,

 

If you want to overwrite changes, then this is not something that an incremental refresh can do. To mitigate this, it is your responsibility to determine if and when to refresh old partitions or perform a full refresh across all partitions. Also, verify query folding; model refresh queries may be unfolded, resulting in long refresh times and high query hybrid engine resource utilisation. This happens because a query is created for each partition in the model. If the query is not collapsed and the data is not filtered in the data source, the engine will attempt to filter the data.

 

Here are some links as well as a solution that will hopefully help you.

Solved: Incremental Refresh taking much more time - Microsoft Fabric Community

Troubleshoot incremental refresh and real-time data - Power BI | Microsoft Learn

 

If your Current Period does not refer to this, please clarify in a follow-up reply.

 

Best Regards,

Clara Gong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Thank you very much for youe reply.

Yes, as you are mentioned we will perform full refresh to handle updates to older data.

With regarding to Inc.refresh, i can see the native query.
select [_].[Number],
[_].[CreatedDateTimeStamp]
from [SSO].[vw_SSBI] as [_]
where [_].[CreatedDateTimeStamp] >= convert(datetime2, '2024-04-01 00:00:00') and [_].[CreatedDateTimeStamp] <= convert(datetime2, '2024-04-02 00:00:00')

 

anything i need to change?

 

If you see i'm getting just 2 columns. One is number and other one createddate used in inc.refresh setup.

 

ShaShaSad_0-1714639212503.png

 

just 2 columns taking so long!

 

ShaShaSad
New Member

From the same DB, I have set up inc.refresh on another table --> It's working fine. 

Full refresh: 1Hr

Inc refresh: 6 mins.

 

not sure why one table inc.refres working fine and for another table no.

 

ETL on Probelm table: Updates and Inserts

ETL on good table: Only Inserts

 

Incremental setup on both tables is on createddate.

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.