Forum Discussion
Dataflow Gen2 Issue
Hi,
I'm trying to run a very simple Gen2 Dataflow from an on prem sql server.
All setups fine but when I ask it to refresh I get:
"Error: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: We don't support creating directories in Azure Storage unless they are empty."
Thanks
Ben
20 Replies
There was a new version of the Gateway released today (May 30th). The version number is 3000.174.12. That version addresses the "We don't support creating directories in Azure Storage unless they are empty" error.
I've updated to 3000.174.12.
This has improved things in that the Gen2 Dataflow will now refresh when a destination is not specified. However if I ask it to write to the lake house I now get:
“null Error: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Microsoft SQL: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) Details: DataSourceKind = Lakehouse;DataSourcePath = Lakehouse;Message = A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server);ErrorCode = -2146232060;Number = 53;Class = 20 GatewayObjectId: 2bcf3e12-740c-47e3-86f3-b94626567200. .”
I wondered if I needed to configure the lakehouse connection to use the gateway but it won't let me do that.
Ben
When you're editing your dataflow, are you able to see the data from your SQL Server in there? Does your Server require IPs to be in an allow list? could it be the firewall at your data source level preventing the connection?
We are getting error while writting to lakehouse, source is onprem oracle databse, with out writting to lakehouse the gen2 dataflow refreshes succesfully.
Mashup Exception Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Microsoft SQL: A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.) Details: DataSourceKind = Lakehouse;DataSourcePath = Lakehouse;Message = A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.);ErrorCode = -2146232060;Number = 121;Class = 20 GatewayObjectId: cd3ddd2d-fd2e-43c2-b2eb-d5f50a748335
this could potentially be a firewall rule set in your environment that is preventing the communications between the gateway and one of the services required to run the dataflow. Please check the article below:
Configure proxy settings for the on-premises data gateway | Microsoft Learn
If this doesn't solve the scenario, please create a new topic / thread in the forum so that we can better understand your scenario.
Hey Ben!
What version of the gateway are you using? We recently released a new one (on Friday) that should solve cases like this one.
If the last version is still giving you issues, please reach out to our support team so they can further assist.
I think I might be running the latest version (the number is higher than what is listed in the blog):
Hi miguel ,
Really odd, so having checked I last updated my gateways on thursday which had resulted in the 3000.178.3 version shown in the screen shot. I've now downloaded the latest version from Microsoft which it initially wouldn't let me install because it was a lower number. Uninstalling everything and rebuilding the cluster I'm now running 3000.174.11 which is the latest "correct" version I can see.
Unfortunatly the error still persists. How do I contact your support teams?
Thanks
Ben
Hey!
You can click on the get help option inside of Microsoft Fabric. For your convenience, I've also left the direct link below:
https://support.fabric.microsoft.com/en-US/support
it's the "create support request" for Data Factory
Arentir not yet. However having put a ticket into Microsoft I got a phone call this afternoon and have a teams call with an engineer tomorrow afternoon (3pm uk time).
I'm fairly certain it's an issue with the mashup engine inside the on prem gateway. (Ran sql server profiler and can see it grab the data fine).
Will update when I have new info!
Microsoft have confirmed it's a bug and they're working on it.
Hi bcdobbs . Is this scenario working reliably for you now, after the extra (undocumented) firewall change? If not, would you mind please sharing the Microsoft ticket #?
We are running into a very similar issue with the latest gateway. In our case the error disappears with very small data volumes.
Thank you!
Hey!
please check the documentation that we've recently made available a couple of days ago:
https://learn.microsoft.com/en-us/fabric/data-factory/gateway-considerations-output-destinations
In addition to what's covered in the article (authorizing *.datawarehouse.pbidedicated.windows.net), port 1433 needs to be opened. We will be addressing the documentation gap shortly.
Thanks
