Dataflow refresh started failing today MashupException.Error

You can click this

to go into more details, hopefully it has the error text.

A refresh from Jan is too old for us to look into anyway (we only maintain logs for 28 days). If you can still reproduce this error, please do the following:

1. Turn on Additional Logging on the Gateway to obtain Mashup logs, then refresh and produce this error, then export the logs

2. Send me the logs and request ID of the failure.

Also, I should add that of the cases of "104100 Couldn't refresh an entity because of an internal error", they were all because the GW server could not establish connection to the SQL endpoint of the destination LH.

We'll need GW logs to confirm that, but you can quickly try this on the GW server too: make a TCP connection to the LH SQL endpoint, port 1433, and see if that's successful. If not, then you are inside some kind of firewall that has allowlisting only for staging lakehouse and not for the output.

Yes please, turn on GW logging is critical to see the real issue. Also, for this particular error, the correlation is a "session ID" from the Options -> Diagnostics dialog.

Pinging is not enough, you'd probably at least want 

Test-NetConnection [address] -Port 1433

As we have seen with other reports, there may be other issues preventing a successful connection, such as:

- TLS issues

- Certificate validation issues

- Issues not related to SQL endpoint, such as getting to the management endpoints like api.powerbi.com.

The GW additional logging will reveal the exact problem limiting the connectivity.

and no, old OnPrem SQL version does not prevent you from connecting back to Lakehouse

Sounds good - please make sure the additional logging is on, and THEN try to repro, capture the session ID with that repro so we can correlate with the logs. When the repro session is over, close the editor and then go export the GW logs

pqian_MSFT What happens now, all the previous errors do not occur because I am now stopped in the 'Choose destination target' dialog with the following error:

To be clear: 

0__create DF

1__connect to on-prem source (data comes in, can see table)

2__disable staging on the query

3__in Data destination, click + icon and select Lakehouse

4__in 'Choose destination target' dialog, select the lakehouse and click next (validation ensues for a couple of minutes)

5__error message above.

Do you still want me to proceed with getting the GW Mashup logs or is this enough info to start a different exploratory path?

pqian_MSFT my sysadmin added these endpoints to the firewall rules:

• Protocol: TCP
• Endpoints: *.datawarehouse.pbidedicated.windows.net, *.datawarehouse.fabric.microsoft.com, *.dfs.fabric.microsoft.com
• Port: 1433

as per the MS documention:  On-premises data gateway considerations for data destinations in Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

For some reason, my sysadmin said there is an issue with the last endpoint: *.dfs.fabric.microsoft.com. He couldn't raise it. Could there be some other endpoint to add to the firewall rules?

But he was able to ping the LH SQL endpoint:

pqian_MSFT btw, could the old version of SQL Server we are using be an issue?

Just sent you the Session ID in PM from my last try.  sysadmin is busy atm, but will get to it today... i will need an upload link though, PM editor has not attach file button.

Understood.  As soon as my sysadmin becomes available again, he'll do this today and I'll send the results+logs.