topic Re: How to debug a data pipeline, How to run the data pipeline up to an activity in Pipelines

How to debug a data pipeline, How to run the data pipeline up to an activity

Srintata — Fri, 01 Mar 2024 10:11:50 GMT

Hi,

I could not find a way to put a breakpoint in datapipeline.

for example, I have 3 copy activities in the data factory pipeline, and I want to run till the second copy activity, to debug and check data.

In Fabric data factory I could not see a way to put a breakpoint same as Azure data factory.

Please help me with the best way to put the break point to run until a specific activity.

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

Anonymous — Fri, 01 Mar 2024 10:35:14 GMT

Hi Srintata
Thanks for using Fabric Community.
You can use the deactivate option in the pipeline. Just right click on the required activity and select deactivate.

This makes the particular activity inactive and you can easily debug in this way.

Hope this helps. Please let me know if you have any further questions.

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

Srintata — Fri, 01 Mar 2024 14:45:13 GMT

Anonymous , Thanks for your reply,

I checked it , when we deactivate , it is just skipped, all the subsequent activities are running.

In Azure data factory, we get a red circle at right top corner to stop the flow at the activity. We use that to run until the required activity check the output till that activity and then run from then on.

Here in Fabric DF, we dont see that red circle at top right of the activity to stop the flow at the activity.

Please help with with such break point in Fabric DF.

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

Anonymous — Fri, 01 Mar 2024 18:09:04 GMT

Hi Srintata
I understood your ask and I know about the breakpoint option in Azure Datafactory. But that option is not available in Fabric and hence I mentioned a work around. So wherever you want to stop the flow of the activity , deactivate all the items from that activity. This is similar to breakpoint in Azure Datafactory.

You can check the output after the second copydata . If its correct then you can rerun the whole pipeline. This is a work around option.

If you want to ge the breakpoint feature in Fabric, please share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Hope this helps. Please let me know if you have any further queries.

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

Anonymous — Mon, 04 Mar 2024 07:02:21 GMT

Hi Srintata
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

Anonymous — Tue, 05 Mar 2024 17:22:58 GMT

Hi @Srintata
We haven’t heard from you on the last response and was just checking back to see if your query got resolved. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

smpa01 — Fri, 26 Jul 2024 15:08:10 GMT

Anonymous is debugging feature going to be added later?

Re: How to debug a data pipeline, How to run the data pipeline up to an activity

JasleenKaur — Thu, 12 Dec 2024 17:00:32 GMT

Hi, thanks for sharing the solution. However, when we run debug mode in synapse pipeline, we can see data preview as well while we are performing the steps within an activity. Is it possible to see that in fabric pipeline?