Get Metadata activity doesnt terminate sometimes

[shoefried]

In our datafactory solution we have a pipeline which uncludes a few "Get Metadata"-activities. The pipeline runs every 30minutes and sometimes it fails (around 1/70 of the runs) because one of the "Get Metadata"-activities doesnt terminate. Subsequently, the outer-activity and/or the pipeline fails. The activity which is not terminating as not always the same andtherefore doesnt access the same folders in a ADLS Gen 2 always, but they all access the same DataLake. I checked the logs and it seems there are also logs regarding the termination of the activity( no "Error"- and no "Succeeded" logs). When i monitor the pipeline runs it says the activities are still running since days even thouigh the pipeline terminated already. The reasons why the pipeline failed are either:  "Operation on target TestDimTableLoop failed: Activity failed because an inner activity failed" if the "Get Metadata"-activity is within a loop and "InternalServerError executing request" if not, but I dont see anything more and dont know how to look any deeper into that. Does anyone might have an idea what the problem might be?

[lbendlin · Accepted solution]

If you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Otherwise you can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues .

[v-hashadapu]

Hi @shoefried , Please let us know if your issue is solved or if you have raised a Microsoft Support ticket. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.

[v-hashadapu]

Hi @shoefried , Please let us know if your issue is solved or if you have raised a Microsoft Support ticket. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.

[v-hashadapu]

Hi @shoefried , Please let us know if your issue is solved. If it is, consider marking the answer that helped 'Accept as Solution', so others with similar queries can find it easily. If not, please share the details.
Thank you.

[v-hashadapu · Accepted solution]

Hi @shoefried , Thank you for reaching out to the Microsoft Community Forum.

This issue likely stems from ADLS Gen 2 throttling, network issues, or ADF service timeouts. To fix, add a timeout and retry policy (3 retries, 30s delay) to each activity, check ADLS metrics for throttling or errors in Azure Monitor, ensure your Integration Runtime is healthy, and enable detailed ADF logs via Diagnostic Settings.

If the above doesn’t resolve your issue, contacting Microsoft support and raising a support ticket with them is the best solution as suggested by @lbendlin . Make sure to provide them with all the troubleshooting steps you've already taken to help them understand the issue better and provide a solution.

https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket

If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.

[lbendlin · Accepted solution]

If you have a Pro license you can open a Pro ticket at https://admin.powerplatform.microsoft.com/newsupportticket/powerbi
Otherwise you can raise an issue at https://community.fabric.microsoft.com/t5/Issues/idb-p/Issues .