Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
D_PBI
Post Patron
Post Patron

ADF Pipeline to extract from a Power BI semantic model (formally a PBI Dataset)?

Hi,
Can an Azure Data Factory Pipeline extract data from a Power BI Semantic model (formally a PBI Dataset) and load that data to an Azure SQL Server database?
I don't have a Data Factory instance installed yet and therefore don't have the option to navigate the interface to see what source connections are available.
If it is yes to my question, is there a link one can provide for me please?
Thanks.

1 ACCEPTED SOLUTION

you can use the Power BI gateway to extract data from On Prem database in case if you are using Fabric Data pipelines.

In case if you are using Azure data factroy, you would need to install SHIR on the on prem server




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com

View solution in original post

3 REPLIES 3
NandanHegde
Super User
Super User

Azure Data factory is similar to Fabric data pipelines.

The below link provides the ask w.r.t ADF which you can follow for data pipelines:

https://datasharkx.wordpress.com/2023/03/02/overcoming-data-size-row-limitations-of-power-bi-rest-ap...

 

Note : we are basically using the Power BI Dataset REST API




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com

@NandanHegde- thanks for your reply. I have a more specifc question to ask on retriving external on-premises data.


An external company holds financial data in an on-premises SQL Server database.
We want to bring that data into our domain. To do this currently, we have a Power BI Semantic Model (formally a Power BI Dataset) that extracts, via a Power BI Gateway (that is installed on the third-party's server), the external financial data.
The result is the Power BI Semantic Model, held within our domain, now contains the financial data.
A simple path is:
On-premises SQL Server database (external) >> Power BI Gateway (external) >> Power BI Semenatic Model (internal)

We are now looking to use Azure Data Factory Pipelines to extract this same external financial data and store it in an Azure SQL Server database within our domain.
To do this, we need the third-party to install a Self-Hosted Integration Runtime (SH-IR) on their server so we can access their data. This SH-IR is the access feature that serves the same function as a Power BI Gateway for the method described above.

To my question, is there a way the Azure Data Factory Pipeline can extract external financial data from the third-party using the existing Power BI Gateway that is already installed on the third-party server?
So the intention will be:
On-premises SQL Server database (external) >> Power BI Gateway (external) >> Azure Data Factory Pipeline (internal) >> Azure SQL Server database (internal)

I need to understand if we can re-use the Power BI Gateway with Azure Data Factory (ADF) Pipelines or if we have to use an SH-IR when using ADF Pipelines?

Thanks.

you can use the Power BI gateway to extract data from On Prem database in case if you are using Fabric Data pipelines.

In case if you are using Azure data factroy, you would need to install SHIR on the on prem server




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

November Update

Fabric Monthly Update - November 2024

Check out the November 2024 Fabric update to learn about new features.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.