topic Re: On Prem SQL Server to Fabric Warehouse via Copy Data in Pipelines

On Prem SQL Server to Fabric Warehouse via Copy Data

MOVC — Wed, 03 Apr 2024 12:44:43 GMT

I've been testing the Copy Data activity from On Prem SQL server via a Data Gateway which was recently made available. I have successfully managed to load into a Lakehouse, but when trying to get Data into a Warehouse it requires that staging is enabled but does not support Workspace Staging and needs a connection to an Azure Environment. I'm assuming this is intentional, but are there any plans to allow this to use the StagingWarehouse objects in the future? It would be nice to keep everything in OneLake which would seem to fit with it's described purpose.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

ShaunBrewer — Wed, 03 Apr 2024 13:16:53 GMT

Good question - I would like to know the answer too.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

Joshrodgers123 — Wed, 03 Apr 2024 15:43:26 GMT

I posted the same thing yesterday. Seems to go against the "OneLake" concept to have to bring additional Azure resources to copy to a warehouse.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

Anonymous — Thu, 04 Apr 2024 05:55:38 GMT

Hi MOVC ,

Thanks for using Fabric Community.

A simple workaround is by loading this to a cloud Lakehouse first, and then another copy activity to write that to WH.

In between we are also reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

Anonymous — Mon, 08 Apr 2024 11:25:11 GMT

Hi MOVC , Joshrodgers123 , ShaunBrewer

Internal Team confimed that, it is on the roadmap. At present we don't have any ETA. But we can expect this feature in future.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

Anonymous — Mon, 08 Apr 2024 18:50:48 GMT

Hi MOVC ,

Glad to know that your query was resolved. Please continue using Fabric Community on your further queries.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

suren_007 — Thu, 08 Aug 2024 23:59:47 GMT

Hi - is there any update to this?
Copying from on-prem -> Lakehouse -> Warehouse isn't a practical solution for us. Is there an alternative? ( sorry am new to Fabric ).

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

smeetsh — Tue, 20 Aug 2024 01:56:23 GMT

One work around may be to use the gateway to get the information, but in stead of writing it to a table, you write it to a scv file, in the next activity you read that csv file and ingest into the warehouse. We ingest several csv files that way.

This way the lakehouse is only used to store a file, and nothing else.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

jagdishwagh0909 — Wed, 08 Jan 2025 13:46:26 GMT

Currently, it is not possible to directly copy data from an on-premises SQL Server (via the gateway) to a Fabric Warehouse. The recommended workaround is to first copy data from SQL Server to a Lakehouse (copy actiovity 1)and then move it from the Lakehouse to the Warehouse (Copy Activity 2). However, this is not an optimal solution, especially for small datasets, as it introduces unnecessary complexity and inefficiency.

We request the Fabric team to address this issue as soon as possible. Delaying this feature may lead to significant rework in the future if direct data transfer from SQL Server to the Warehouse is eventually enabled.

Re: On Prem SQL Server to Fabric Warehouse via Copy Data

vasalari — Tue, 18 Feb 2025 23:00:49 GMT

Hello and so many thanks. There are links that indicates it is posssible This link actually explains how to do it. Microsoft Fabric: Import On Premise SQL Server 2022 Data into Microsoft Fabric with Data Pipelines!! Can you kindly take a look? Sincerely appreciate your help