Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
suidino2
Frequent Visitor

Dataflow problem with appending to existing table

Dataflow gen 2 with On Premise SQL is troublesome. 

 

The scenario is that

1. I have a Dataflow Gen 2 (DF2) which loads data from on premise SQL (by using gateway the latest version) to lakehouse for those data from year of 2010 to 2015. This DF2 has no problem. 

 

Then,

 

2. I created another DF2 using the same M code to load data from the same on premise SQL table for data fitlered between year 2016 to 2020 to existing lakehouse table created by previous DF2. But I got this error.

 

"Null Error: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: An error happened while reading data from the provider: 'A transport-level error has occurred when receiving results from the server." 

 

Before I reach to this scenario, I have also encountered gateway issue and firewall network issue described in other thread. 

The overall impression is that I felt a bit unstable using DF2 especially when it needs to work with on premise SQL....

And other than DF2, we dont have any other option to laod data from on premise to Fabric (Lakehouse), it feels stressful.

Hope MS can develop and enable connector of on premise SQL in the pipeline Copy Activities soon. 

2 REPLIES 2
suidino2
Frequent Visitor

It turned out that it wa because I didnt pay good attention to the message.

When pulling data from source and to create a table in Lakehouse, Lakehouse will automatically change the column type to whole number if the column has only 0 value even you set the column type as decimal. Because of that, the second append dataflow will cause a column type inconsistency. Meaning this particular column in your dataflow is set as decimal type, but the existing table in lakehouse is actually whole number. Coumn type inconsisency. this was the main issue cuasing the error. 

So, eventually instead of 

SidJay
Employee
Employee

Could you please try upgrading to the latest Gateway (version 3000.178.9 released on June 15th). That should help address at least one class of errors involving output destinations and/or queries that depend on other staged queries.

 

Please let us know if the issues persist after that. Thanks.

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Fabric Monthly Update - February 2024

Check out the February 2024 Fabric update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors