Invoke Pipeline fails with Error code 2015: 'Value cannot be null. Parameter name: source '.

[AnitaCrook]

We have a parent pipeline with a Lookup and a ForEach loop.  The Lookup reads a list of tables from an ETL Control table in a Fabric SqlDb database, then inside the ForEach loop, calls a child pipeline with a single Copy Activity.  The Schema and Table names to be copied are passed from the output of the Lookup to the child pipeline as parameters.

The child pipeline fails consistently with this error:  Fabric activity failed. 'Value cannot be null.  Parameter name: source '.

As a workaround, we are able to use the Legacy Invoke Pipeline successfully.  It is configured identically to the newer Invoke Pipeline.

[AnitaCrook]

Hi @svenchio,

For now, I'm only processing a couple dozen tables in the loop as a POC; eventually we'll have 400 tables to process.  The Copy Activity inside the ForEach loop uses DB2 as the Source, and there's an existing Issue related to DB2 source connections; perhaps that's related to my error?  
Outer pipeline:

Inner pipeline:

[svenchio]

Ohhh wow @AnitaCrook , seems like you having a big data process 👉  400 tables! 👏😎  I don't have a DB2 source to even try to reproduce your issue, I guess that won't be a surprise to you, I bet you're on a middele of a platform modernization to either move or make the data accessible to Fabric (clarify if neccesary for this case)... back to the case, I would be suspecious of the DB2 connector, of course, but this would not explain why using using the legacy invoke pipeline works! Both are calling the exact same Inner pipeline, that means, both are calling the exact same DB2 connectino and therefore, if one works and the other don't, this suggest that is not the DB2 per se, but perhaps the Invoke pipelie and it's interaction with the inner pipeline. So far, some test that I've done works just fine ... but in your case, this has to be reproduce with a DB2 connector to figure this one out ... I suggest you to at least try to identify if there any pattern on the error and open a case with MSFT.

Hope at least this brainstorming session helps in the resolution of the issue ... kudos if you find this useful. All the very best!  

[v-achippa]

Hi @AnitaCrook,

Thank you for reaching out to Microsoft Fabric Community.

Thank you @svenchio for the prompt response. 

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[v-achippa]

Hi @AnitaCrook,

We wanted to kindly follow up to check if the solution provided by the user for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

[AnitaCrook]

Hi @v-achippa , 

No solution yet, the only way we can invoke a pipeline which copies from DB2 as a source is to use the Legacy Invoke Pipeline.

[v-achippa · Accepted solution]

Hi @AnitaCrook,

Currently there is no additional setting in the new Invoke Pipeline activity that can be changed to resolve this. The behaviour you are seeing looks like a service side issue that affects DB2 sources when invoked through the new activity.

For now, please continue using the Legacy Invoke Pipeline, as it works for your DB2 copy scenario.

We recommend opening a Microsoft Fabric support ticket with the failed run id's and pipeline details so the product team can investigate and address this issue.

To raise a support ticke, kindly follow the steps outlined in the following guide:

Create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn

Thanks and regards,

Anjan Kumar Chippa

[v-achippa]

Hi @AnitaCrook,

As we haven’t heard back from you, we wanted to kindly follow up to check that have you raised the support ticket? Is your issue resolved?

Thanks and regards,

Anjan Kumar Chippa

[svenchio]

Hi @AnitaCrook  this case seems very interesting, I don't have an answer for you because I don't think I have ever seen this behavior but I can certanly try to replicate it on my environment and see what I can find! 

How many tables are you processing during the loop (e.g. dozens, hundreds, ?), have you observed any pattern in the error? I guess the copy activity varies in time, for instance, some copy activities take seconds, some of them minutes?  I'll try to do some codign meanwhile and share my observations, all the best and good luck.