Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Datamart incremental refresh, it does full refresh even with data alredy loaded

Trying to do incremental refresh on a new  Datamart. Setup is refresh period is 10  days, keep data from last 50 years. Datamart create 20 or more SQL queries to my SQL server, getting wrong periods of date (CONTROLE is the key date column). Refresh only if max Controle changed. Datamart had already refreshed first time (has already data inside dataflow):

 

Jrussi1301_0-1679352406799.png

Jrussi1301_1-1679352622991.png

 

 This happens to all my 30 tables. Refreshing process takes more than  hours, and sometimes it fails (error 500). Normaly the same refreh is 30 minutes on my SSAS, full refresh. What I see is that it does a full refresh, at all tables, even if it already has data loaded, after the initial refresh.
It happens if I create a new colunm in one table, or change/create an incremental refresh setup. Even doing it to one table, it full refreshes all tables.
The dataflow interface on a Datamart is terrible. Each change you do in one table starts a full refresh of the others tables too, even if they have already an incremental refresh setup, and data loaded after a first refresh. And you can't edit other table before this full refresh.

With old dataflow, you can edit everything before starts a refresh, that can be a booked one, other day or hour.

Lost a lot of time and finally quit, transfering to a regular dataflow. I regret to have started  a datamart. My intention was to use the SQL capabilities, but forget it, until it is stable I will not try again.

Status: Investigating

Hi @Jrussi1301 ,

Using incremental refresh in dataflows created in Power BI requires that the dataflow reside in a workspace in Premium capacity.

Then check if the configuration setted correctly:

Using incremental refresh with dataflows - Power Query | Microsoft Learn

Verify if the datasource is supported and query folding is required, refer to:

Incremental refresh for datasets and real-time data in Power BI - Power BI | Microsoft Learn

 

Best regards,

Community Support Team_kalyj

Comments
v-yanjiang-msft
Community Support
Status changed to: Investigating

Hi @Jrussi1301 ,

Using incremental refresh in dataflows created in Power BI requires that the dataflow reside in a workspace in Premium capacity.

Then check if the configuration setted correctly:

Using incremental refresh with dataflows - Power Query | Microsoft Learn

Verify if the datasource is supported and query folding is required, refer to:

Incremental refresh for datasets and real-time data in Power BI - Power BI | Microsoft Learn

 

Best regards,

Community Support Team_kalyj

Jrussi1301
Frequent Visitor

I use a PPU license. Connecting to a SQL server without using native SQL query (pure M language in Power Query).