Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
S4CSurrey
Helper I
Helper I

When a dataflow refresh fails, what happens to the data in the entities?

When a dataflow refresh fails, what happens to the data in the entities?

 

Is the data only partially refreshed, or does the new data not come in at all?

 

I ask because after a failed dataflow refresh, I look at my report, and the data shown is cut off. This is because on my API end, there has been a timeout, which leads to an error in the dataflow, which results in a failure.

 

How do I instead stop the new data from coming in entirely? I don't want partial data.

7 REPLIES 7
Greg_Deckler
Super User
Super User

@S4CSurrey , unless I am mistaken, I believe the order of operations is:

 

1. Drop all rows in table

2. Load rows

 

So, it is adding rows up until the point of failure. So, if it fails after 10 rows are imported, you would have 10 rows. If it fails after 100 rows are imported, you would have 100 rows.


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

Alright, that's what I thought might be happening!

 

So in that case, how would I work around this issue? Ideally, I'd like to maintain the data that was previously loaded in a working refresh, and only bring in what was updated in the new refresh? The issue is, if the new refresh errors out, it would have cut off potential new data.

Hi @S4CSurrey ,

 

When you fix the refresh failure problem and refresh again, the data refreshed for the first time will be overwritten.
You may try to refresh again and then check the dataset.

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

That's the issue, I can't tell the future, I don't know when and if the data refresh will ever fail again. Due to that, the new "failed" refresh will bring in partial or empty data, overwriting my dataset. How do I prevent this?

Hi @S4CSurrey ,

 

If you receive a notification that the refresh failed, maybe you can refresh it again manually.

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

The data refreshes automatically 20 times a day. It's not feasible to manually intervene and run a manual refresh on every automated refresh.

There has to be some better error handling options in Dataflow!

Hi @S4CSurrey ,

 

This is a good idea, please submit your idea .

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Power BI Header Change

Power BI Header Update

Read about the exciting updates for the Power BI forum.

Top Solution Authors
Top Kudoed Authors