Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn the coveted Fabric Analytics Engineer certification. 100% off your exam for a limited time only!

Dataflow issue

Hi team,

 

I am trying to use dataflow on premium capacity to store daily snapshots from an SQL server. Basically, the data required is to get visibility on some ticketing tool backlog (open tickets at the end of the day). The SQL does have a column date, that is the getdate() for when the SQL is running. 

 

I have configured the below incremental refresh in the dataflow. My expectation is to have data for each day when the SQL is running, however, i only have the last day. Can you please help me with some hints on how i should configure this dataflow?

 

Many thanks!

Theo

 
 
 
 

Capture.JPG

Status: New
Comments
v-qiuyu-msft
Community Support

Hi @Anonymous, 

 

In your scenario, when you define the incremental refresh policy, please use a different column to define the Detect data changes condition. See: https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#detect-data-changes

 

q5.PNGq4.PNGq6.PNG

 

Best Regards,
Qiuyun Yu

Anonymous
Not applicable

Hi Qiuyun,

 

Thanks for your reply. In my use-case, the ticket can be found multiple times in the dataflow result. That is the expected outcome. Since i need to have backlog, then on day 1, ticket id 1, because it's open, will be in the backlog. On day 2, ticket 1 is still open. Due to the incremental refresh, the dataflow should contain twice the ticket id 1, for each day when the ticket was open.

 

I will test the above recommendations and see if this will work.

 

Thanks,

Theo

Anonymous
Not applicable

Hi @v-qiuyu-msft ,

 

I have performed the changes you suggested and i still only get the last day of data and the previous one is deleted. Any idea on how i can resolve this challenge please?

 

Many thanks,

Theo

v-qiuyu-msft
Community Support

Hi @Anonymous, 

 

I would suggest you create a support ticket to let engineers look into the issue on your side. 

 

Support Ticket.gif

 

Best Regards,
Qiuyun Yu

Anonymous
Not applicable

Many thanks @v-qiuyu-msft , I will do that. have a good day!

Anonymous
Not applicable

Hi again,

 

@v-qiuyu-msft , while checking my dataflow before submitting the ticket, i just saw that i have 2 days of data (expected outcome), which means that for now everything seems to be fine. I will continue to monitor it and only raise ticket if needed. Many thanks for all your guidance and help.

 

Have a good day,

Theo