topic Re: Dataflow Gen2 (CI/CD preview) - can not connect to lakehouse in Dataflow

Dataflow Gen2 (CI/CD preview) - can not connect to lakehouse

n5s5n — Tue, 04 Feb 2025 07:47:26 GMT

Hi everyone

I am trying to use a Dataflow Gen2 to get data from a Lakehouse table, process it, and then store the data in another Lakehouse table.

When I created a new Dataflow, I noticed that the type had changed from Dataflow (Gen2) to Dataflow Gen2(CI/CD Preview).

When I tried to pull data from the Lakehouse in a Dataflow Gen2(CI/CD Preview), I got an error with the connection.

In this case, select No Data Gateway and Organizational Account as the authentication type.

The error message is translated into English as follows:
"An exception occurred: The call to the downstream service for URL 'https://api.powerbi.com/powerbi/globalservice/v201606/clusterdetails' failed. Error code: 'DataSource.Error' and error message: 'Unable to connect to remote server.'. (Session ID: ebb51344-05b2-4345-9ea9-8a3c53b97b9e, Region: japan)"

A similar error can occur when connecting data to a destination.

The previous Dataflow had no problem connecting to the lake house and reading and writing data from the tables.

If you have any suggestions on how to resolve this error or how to create a traditional Dataflow(Gen2) that is not a CI/CD preview, please let me know.

Thanks,

n5s5n

Re: Dataflow Gen2 (CI/CD preview) - can not connect to lakehouse

coverturfLSH — Tue, 04 Feb 2025 17:35:02 GMT

I have the same issue as you in not being able to publish to a Lakehouse. I was able to fix it by unchecking the "Enable Git integration, deployment pipelines and Public API scenarios (preview)" when creating a new DFG2.

Re: Dataflow Gen2 (CI/CD preview) - can not connect to lakehouse

n5s5n — Wed, 05 Feb 2025 00:20:29 GMT

Hi coverturfLSH

Thank you for your kind response. I really appreciate your help.

When I created a Dataflow Gen2, "Enable Git integration, deployment pipelines and Public API scenarios (preview)" was grayed out and I couldn't uncheck it.

However, when I created the Dataflow Gen2 in the workspace root, I was able to uncheck it and then move it into my project's folder.

After that, I successfully retrieved data from the Lakehouse using the traditional method.

Thanks again

n5s5n