Data Pipeline connection error

[schneiw]

Hi

I have a managed connection that is using the on prem data gateway. If i test it from the Manage Connections screen in Fabric it gives no error. I used it in a Gen 2 dataflow, no problem. 

But, if i add it to my Copy activity in a data pipline and test or try to use it there, i get this error: 

The integration runtime '16456761-edc2-46f5-aebe-eef1ea45609e' is not registered or has expired.

I have looked everywhere I can think of but no one seems to have reported this issue. Or am I missing something? Not sure how to troubleshoot it.

Thanks!

[schneiw]

Thanks for the replies. My connection was a simple on prem SQL Server database. It turns out my DEV gateway was a higher version (latest) than what was on TEST and PROD. My DEV environment worked, but TEST and PROD did not. After I upgraded the TEST gateway it worked. I will do the same on PROD, but we have a strict release process so I can only do that next week, but based on TT enviroment, the upgrade to gateway will fic the issue. Thanks @yvaubourg for the recommendation to try the verion upgrade.

[v-karpurapud]

Hi @schneiw 

Welcome to the Microsoft Fabric Forum. Also thank you @yvaubourg and @Srisakthi for your quick responses

The error stating that the integration runtime is "not registered or has expired" in your Microsoft Fabric data pipeline typically occurs due to misconfigurations or unsupported scenarios specific to pipeline activities, even when the same managed connection works in Dataflows Gen2. Although the gateway connection may test successfully and work within Dataflows, pipelines require additional configuration. 

It is important to confirm that the data source involved is supported specifically for use in data pipelines, as compatibility varies between pipelines and Dataflows. Microsoft provides a list of supported on-premises connectors to guide this verification. Additionally, successful operation of pipelines requires that the on-premises data gateway be updated to the latest version, and that the associated integration runtime be correctly registered and recognized within the Fabric workspace. 

Network configuration also plays a crucial role; the machine hosting the gateway must be able to establish outbound HTTPS connections to *.frontend.clouddatahub.net, as this is essential for the integration runtime to communicate with Fabric services. 

Moreover, data pipelines do not support accessing local resources on the gateway machine itself only network-accessible or remote data sources are supported. Ensuring alignment with these requirements will enable successful execution of the Copy activity within the data pipeline.

If this information helps resolve your issue, kindly consider marking this response as the Accepted Solution, as it may assist other community members facing similar challenges.

 

Thank you for being part of the Microsoft Fabric Community.

[Srisakthi]

Hi @schneiw 

As @yvaubourg  mentioned not all the sources supported by dataflow gen 2 is supported in data pipeline.

Here you can find the list of connectors(on-prem) supported in data pipeline

https://learn.microsoft.com/en-us/fabric/data-factory/how-to-access-on-premises-data

Also you have to make some changes for data pipeline to work like network settings

You need to configure the firewall to allow outbound connections *.frontend.clouddatahub.net from the gateway for Fabric pipeline capabilities.

Note : Local access to the machine with the on-premises data gateway installed is not allowed in data pipelines.

Regards,

Srisakthi

If this answer solves your question, please mark "Accept as Solution.

[yvaubourg · Accepted solution]

Hi,

In my experience, copy data in pipelines with data gateway don't work with all sources enven if it work with Dataflows. Wich Data Source do you have?

Somenone here say that updating your gateway to the latest version can work.