Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
suidino2
Frequent Visitor

Dataflow problem with appending to existing table

Dataflow gen 2 with On Premise SQL is troublesome. 

 

The scenario is that

1. I have a Dataflow Gen 2 (DF2) which loads data from on premise SQL (by using gateway the latest version) to lakehouse for those data from year of 2010 to 2015. This DF2 has no problem. 

 

Then,

 

2. I created another DF2 using the same M code to load data from the same on premise SQL table for data fitlered between year 2016 to 2020 to existing lakehouse table created by previous DF2. But I got this error.

 

"Null Error: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: An error happened while reading data from the provider: 'A transport-level error has occurred when receiving results from the server." 

 

Before I reach to this scenario, I have also encountered gateway issue and firewall network issue described in other thread. 

The overall impression is that I felt a bit unstable using DF2 especially when it needs to work with on premise SQL....

And other than DF2, we dont have any other option to laod data from on premise to Fabric (Lakehouse), it feels stressful.

Hope MS can develop and enable connector of on premise SQL in the pipeline Copy Activities soon. 

2 REPLIES 2
suidino2
Frequent Visitor

It turned out that it wa because I didnt pay good attention to the message.

When pulling data from source and to create a table in Lakehouse, Lakehouse will automatically change the column type to whole number if the column has only 0 value even you set the column type as decimal. Because of that, the second append dataflow will cause a column type inconsistency. Meaning this particular column in your dataflow is set as decimal type, but the existing table in lakehouse is actually whole number. Coumn type inconsisency. this was the main issue cuasing the error. 

So, eventually instead of 

SidJay
Microsoft Employee
Microsoft Employee

Could you please try upgrading to the latest Gateway (version 3000.178.9 released on June 15th). That should help address at least one class of errors involving output destinations and/or queries that depend on other staged queries.

 

Please let us know if the issues persist after that. Thanks.

Helpful resources

Announcements
FabricCarousel_June2024

Fabric Monthly Update - June 2024

Check out the June 2024 Fabric update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.

Top Solution Authors