Forum Discussion

cristinaojer's avatar
cristinaojer
Frequent Visitor
2 years ago

Can't load data from SQL Server into Lakehouse. Can't add a data destination.

I'm facing some issues when trying to get data from SQL Server database into Fabric Lakehouse. I'm using an on-premises data gateway updated up to it’s latest version. We’ve tried to get this d...
Need Help
v-cboorla-msft's avatar
v-cboorla-msft
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

Hi cristinaojer 

 

Thanks for using Microsoft Fabric Community. amitchandak thank you for sharing the details.

As we have two releases in the month of April.

April 2024 update (3000.218.8)

April 2024 update (3000.218.7)

 

Request you to install the latest version of on-prem gateway and try to establish the connection, this might help you.

If the issue still persists, please do let us know we will try to help.

 

Hope this helps.

 

Thank you.

brewed's avatar
brewed
Frequent Visitor
2 years ago

Experiencing the sames issues. Dataflow Gen2 successfully loads onprem SQL tables to Fabric lakehouse tables, but a copy activity in a Fabric pipeline dosn't work. I also setup an ADF pipeline to copy to the lakehouse and get the same results.

 

The copy activity is creating the table in the lakehouse as expected. When looking at the underlying files of the table, a folder called '_delta_log' is there with files small json files within it. Looks like each of these files is being created after a timeout occurs? Here's the contents of the json:

 

{"protocol":{"minReaderVersion":1,"minWriterVersion":2,"readerFeatures":null,"writerFeatures":null}}
{"metaData":{"id":"8db4335b-4ae9-4b5e-8262-3e5096d44d39","name":"delta","description":"","format":{"provider":"parquet","options":{}},"schemaString":"{\"type\":\"struct\",\"fields\": <columns names removed>]}","partitionColumns":[],"createdTime":1714817850126,"configuration":{}}}
{"commitInfo":{"operation":"Write","engineInfo":"Microsoft.DI.Delta","additionalCommitInfo":null}}
 
No records are being inserted into the lakehouse. Data preview works in the pipeline.
 
I also tried landing the files as lakehouse files instead of tables, but still did not work. Seems like the issue is with writing data to the lakehouse from a pipeline copy activity (either Fabfic or ADF).