topic Re: ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)? in Pipelines

ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)?

D_PBI — Wed, 29 May 2024 11:38:31 GMT

Hi,
Can an Azure Data Factory Pipeline extract data from a Power BI Semantic model (formally a PBI Dataset) and load that data to an Azure SQL Server database?
I don't have a Data Factory instance installed yet and therefore don't have the option to navigate the interface to see what source connections are available.
If it is yes to my question, is there a link one can provide for me please?
Thanks.

Re: ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)?

NandanHegde — Wed, 29 May 2024 12:32:49 GMT

Azure Data factory is similar to Fabric data pipelines.

The below link provides the ask w.r.t ADF which you can follow for data pipelines:

https://datasharkx.wordpress.com/2023/03/02/overcoming-data-size-row-limitations-of-power-bi-rest-api-for-automated-data-extraction-from-dataset/

Note : we are basically using the Power BI Dataset REST API

Re: ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)?

D_PBI — Thu, 30 May 2024 11:23:50 GMT

NandanHegde- thanks for your reply. I have a more specifc question to ask on retriving external on-premises data.

An external company holds financial data in an on-premises SQL Server database.
We want to bring that data into our domain. To do this currently, we have a Power BI Semantic Model (formally a Power BI Dataset) that extracts, via a Power BI Gateway (that is installed on the third-party's server), the external financial data.
The result is the Power BI Semantic Model, held within our domain, now contains the financial data.
A simple path is:
On-premises SQL Server database (external) >> Power BI Gateway (external) >> Power BI Semenatic Model (internal)

We are now looking to use Azure Data Factory Pipelines to extract this same external financial data and store it in an Azure SQL Server database within our domain.
To do this, we need the third-party to install a Self-Hosted Integration Runtime (SH-IR) on their server so we can access their data. This SH-IR is the access feature that serves the same function as a Power BI Gateway for the method described above.

To my question, is there a way the Azure Data Factory Pipeline can extract external financial data from the third-party using the existing Power BI Gateway that is already installed on the third-party server?
So the intention will be:
On-premises SQL Server database (external) >> Power BI Gateway (external) >> Azure Data Factory Pipeline (internal) >> Azure SQL Server database (internal)

I need to understand if we can re-use the Power BI Gateway with Azure Data Factory (ADF) Pipelines or if we have to use an SH-IR when using ADF Pipelines?

Thanks.

Re: ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)?

NandanHegde — Thu, 30 May 2024 11:35:47 GMT

you can use the Power BI gateway to extract data from On Prem database in case if you are using Fabric Data pipelines.

In case if you are using Azure data factroy, you would need to install SHIR on the on prem server