GEN2 dataflow error - We received multiple duplicate continuation tokens from Azure Storage

[dbeavon3]

Can someone tell me what this means? 

MashupException.Error: Failed to insert a table., InnerException: We received multiple duplicate continuation tokens from Azure Storage.

We have a pretty simple GEN2 dataflow and it suddenly failed today with a meaningless error.  I would like to understand what it means, and why it happens.  I'm guessing it is some sort of regression since I've never seen this before and it seems like the type of thing that we would have encountered by now.

I think this is correlated to a release train that has been sent to our regions this weekend.

Any help would be appreciated.  I have way too many tickets with Mindtree right now and this one is probably going to take two or three weeks of engagement.  They never put their bugs in their "known issues" pages, so I thought I would at least share this here.

[Je-maintiendrai]

We had the same error on 2 of our dataflows this morning after switching capacities, republishing fixed the issue for us. 

[dbeavon3 · Accepted solution]

I spoke with a Mindtree engineer.  The release train went to production on 4/5 and this unusual error appeared to us for the first time in our daily GEN2 refresh on 4/7.

They want me to open a new ticket (SR). After that Mindtree promised to send the bug to Microsoft via ICM.  They claim there are no prior ICM's with a similar error message yet:

We received multiple duplicate continuation tokens from Azure Storage.

I will probably send this issue thru pro support (Mindtree) once our users start to notice/complain.  It may be a few days for them to realize their stuff is broken (again).  The worst part is that this new problem came up in the same "train" that deployed a fix for a different problem.  A very LARGE part of my job these days involves talking to Mindtree support about Fabric bugs.  Some days I find it very troublesome to be a Fabric customer.

[YagizOzd]

This happened to my dataflow gen 2 as well. I am getting this error randomly from any query inside the DF. I have contacted Mindtree engineer but unfortunately we couldn't come up with a solution so far. And the weird thing is they can't see any error code from their side so far.

[v-menakakota]

Hi @dbeavon3 ,

Thank you for bringing this to our attention and for providing detailed context around the issue. We understand how disruptive it can be to encounter unexpected errors, particularly in workflows that have been running reliably.

The error message "We received multiple duplicate continuation tokens from Azure Storage" typically indicates a low-level issue related to data pagination when retrieving content from Azure Data Lake Storage Gen2. While this is not a commonly reported issue, the timing you've identified with the recent release may point to a potential regression.

At this stage, we recommend raising a support request (SR) through your Pro Support channel. This will ensure the appropriate teams can investigate further and take necessary action.

We appreciate you sharing this with the community — raising awareness of such issues is incredibly valuable for all users.

If this post was helpful, please give us Kudos and consider marking Accept as solution to assist other members in finding it more easily.

[dbeavon3]

Hi @v-menakakota 

Thanks for the suggestion.  I will certainly open a pro ticket once the problem becomes more severe.

Can you please tell me the name of the partner organization that you work for?  I have always wondered who works on the Q&A in the forums.

>> raising awareness of such issues is incredibly valuable for all users.

The best way to raise awareness is for Microsoft to do so in their "known issues" page.  Everything we accomplish here in these forums is just guesswork.  Microsoft is in the position to look at their source code, and find the failing component, and describe the reason(s) why this error message would arise.  It is unlikely that there is ambiguity in this error itself, or in the RCA for the error.  It is very likely that the error could be simulated on demand.  Once this happens, the error should be added to the "known issues" page by Microsoft for the benefit of customers.

Are you able to reach out to the Fabric PG team?  Can you please propose to them that they should do the needful?  I think it would be the onelake team.  I have not had any error messages of this nature when using normal Azure storage containers, so it is almost certainly a bug that is introduced by one of the Fabric PG's.

I hesitate to create another SR given that I spend between 100 and 200 hours a year working on cases with Mindtree and I need to be careful about which bugs to invest time in.  Ideally Microsoft would do the heavy lifting for this one, or would add it to the "known issues" page if they don't have a fix yet.

[v-menakakota]

Hi @dbeavon3 ,

Thank you for your interest! Due to privacy and data protection policies, I'm unable to disclose the name of the organization I’m affiliated with. My main focus is to contribute to the community by sharing accurate information and helpful content.We really apologies for the inconvenience.

Regarding the PG team, unfortunately we don’t have any specific details or direct contact with them at this time.

[dbeavon3]

Hi @v-menakakota Seems odd for you to be providing advice in the community without any credentials, or even a company name.  Can you at least say "Authorized Microsoft Partner" or something?  Most of the information that is shared on the Internet is judged based on the source, and the fact that your signature says "community support" implies you are a reliable source so I can't see why you can't confirm it. 

Unfortunately the meaning of "accurate information" is pretty subjective nowadays.  It always depends on the source.  

(If you were to say you work for Mindtree, then it would help me avoid making a duplicate/redundant contact with the "pro" support.  Or, alternately, I might ask that your fellow MT engineers to refer to your advice in the community and they would consider it even more authoritative, given that you both work in the same org.  )

Do you have any process for alerting Microsoft about "known issues"?  That is always a much higher quality source of information about bugs, than the discussion in this community.

I truly appreciate your help, just want to know who I'm working with.

[v-menakakota]

Hi @dbeavon3 ,

We really apologies for the inconvenience, As a community support member, we are here to share guidance based on experience and available resources. We’re kindly asked not to disclose organization names or Microsoft Partner status in the forum, in line with Microsoft’s privacy and community guidelines.

we always recommend that users experiencing persistent or potentially platform-related issues log a support ticket through the Microsoft support channel or if they want to Connect with community members, ask questions, and learn more about Fabric to use the Forum.

Thank you for reaching out to us on the Microsoft Fabric Community Forum.

[dbeavon3]

Hi @v-menakakota 

Earlier you said it "was not a commonly reported issue".  Is this based on a guess, or based on the number of posts in the forum, or based on your visibility into support cases?

Has this bug ever been reported via support ticket (pro support at CSS)?

I'm trying to determine when to report the issue to Mindtree/Microsoft, and whether it might be better to let others spend time on those tickets since I have plenty of other bugs to deal with.

Again, it would be helpful for you to introduce yourself in some way so we can determine what you mean when you claim a problem is not commonly reported.  Assuming you are NOT working for either Microsoft or Mindtree, then you probably don't have any awareness of how common a bug might be.  That information is extremely valuable and most of us don't have a way of knowing one way or another.  I find that even the most common types of problems will not be added to the "known issues" list.

[v-menakakota]

Hi @dbeavon3 ,

We are following up once again regarding your query. Could you please confirm if the issue has been resolved through the support ticket with Microsoft?

If the issue has been resolved, we kindly request you to share the resolution or key insights here to help others in the community. If we don’t hear back, we’ll go ahead and close this thread.

Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.

Thank you for your understanding and participation.

[dbeavon3]

Hi @v-menakakota 
This is still a bug.  I spoke with a PM (Maybe it was Sid?) who recognized the error message.  It is something Microsoft should fix based on their own telemetry.  I haven't reached out to Mindtree engineers to ask them for help yet.  (That is a month-long investment, and I'm hoping another customer will have more appetite for that).

The error message is something they should be able to recreate at will, if they were motivated.  The message comes from their own proprietary code, and is low-level and independent of a customer's specific use-cases.

[dbeavon3]

Hi @v-menakakota 

Can you tell me what org you work for or why you keep following up?  I think it seems likely that this topic is regularly searched, and this page is regularly viewed.  True?

You should know that customers invest far too much time opening SR's with Mindtree.  A ticket like this one would EASILY take me 2 or 3 weeks of effort with Mindtree, and there is absolutely no pot of gold to be found at the end of that effort.  Based on dozens of past experiences it is NOT likely that the PG would quickly fix the bug, nor create a "known issues" page for it.  As such, this is the type of investment I am NOT willing to make right now (until there is an urgent outage that forces me to that point.)

Furthermore, I believe the PG could easily monitor their telemetry, and measure the number of times this issue happens. They could find a pattern and fix it, or simply add it to their known issues page (on their own initiative).  The PG's could take the initiative on something like this if they wanted, and they certainly don't need my help to fix their bugs.

I have an informal contact in the PG and I think the best guess is that there is some sort of error coming out of the lakehouse, when interacting with parquet files.  These implementation details are buried under several layers of abstraction.  Again, it would be easy for the PG to find the error message in their source code, and it would be easy for them to build realistic samples that apply stress to that part of the code.  If they don't want to do all of this work themselves, they should share the source code openly.  Fabric is based on massive amounts of open source code (including all the stuff related to parquet files, so ther is no reason they can't share a little more code and allow the community to find/fix this bug).

[v-menakakota]

Hi @dbeavon3 ,

Thank you again for your thoughtful feedback it's clear you're deeply committed to improving the platform experience, and that’s genuinely appreciated.

When I mentioned this doesn’t seem to be a commonly reported issue, I was referring to what’s visible in the public community space. I don’t have access to internal support data, so my comments are based on community insights only.

For issues like this,if you have Pro license raising a Pro Support ticket is still the most effective way to ensure it gets in front of the right engineering teams. I completely understand the time commitment, but it really helps drive visibility and resolution at the right level.

Thanks again for being an engaged and proactive voice in the community.