Still getting intermittent data refresh failures

Hi Simon,

I have raised a support ticket. 

The pbix files that fail the most are ones with mixed SSAS and SQL server data sources. If we are doing quite a bit of data merging using advanced query editor tools like merging queries these are the ones most affected. Sometimes they work and sometimes they don't. Below is an excerpt from our gateway logs.

This particular fail refreshes no problem if I run it from desktop but it does take about 4-5 minutes to refresh. I never get a timeout when running locally though. It seems this fails in the mornings when we have most of our refeshes scheduled, could it be something on the gateway server when it is processing several refreshes at once.

DM.EnterpriseGateway Error: 0 : 2016-12-07T10:54:21.9607887Z DM.EnterpriseGateway	f1a0e749-489d-480b-ac24-ec3ce3dfff76	80550339-995d-4f24-9f4e-7a05f18d68f7	MGTT	2976f9fa-734c-474e-8398-c5be0a4ea120	6CC65CC9 [DM.Pipeline.Common] Non-gateway exception encountered in activity scope: System.ServiceModel.CommunicationException: The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:01:00'. ---> System.IO.IOException: The write operation failed, see inner exception. ---> System.ServiceModel.CommunicationException: The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:01:00'. ---> System.Net.Sockets.SocketException: An established connection was aborted by the software in your host machine
   at System.Net.Sockets.Socket.BeginSend(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.SocketConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
   --- End of inner exception stack trace ---
   at Microsoft.ServiceBus.Channels.SocketConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.ConnectionStream.BeginWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
   at System.Net.Security._SslStream.StartWriting(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   --- End of inner exception stack trace ---
   at System.Net.Security._SslStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.BeginWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback asyncCallback, Object asyncState)
   at Microsoft.ServiceBus.Channels.StreamConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
   --- End of inner exception stack trace ---

Server stack trace: 
   at Microsoft.ServiceBus.Channels.StreamConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.BufferedConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult.WriteCore()
   at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult..ctor(FramingDuplexSessionChannel channel, Message message, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.OnBeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.Channels.OutputChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
   at Microsoft.ServiceBus.SocketConnectionChannelListener`2.DuplexSessionChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
   at System.ServiceModel.Dispatcher.DuplexChannelBinder.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
   at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartSend(Boolean completedSynchronously)
   at System.ServiceModel.Channels.ServiceChannel.BeginCall(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, TimeSpan timeout, AsyncCallback callback, Object asyncState)
   at System.ServiceModel.Channels.ServiceChannel.BeginCall(ServiceChannel channel, ProxyOperationRuntime operation, Object[] ins, AsyncCallback callback, Object asyncState)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
   at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateTask(ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeTaskService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.PowerBI.DataMovement.Pipeline.InternalContracts.IGatewayTransferCallback.TransferCallbackAsync(Byte[] packet)
   at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<ProcessTransferTelemetryAsync>d__25.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5.<<TransferAsync>b__3>d__a.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__3.MoveNext()
DM.EnterpriseGateway Error: 0 : 2016-12-07T10:54:21.9607887Z DM.EnterpriseGateway	f1a0e749-489d-480b-ac24-ec3ce3dfff76	80550339-995d-4f24-9f4e-7a05f18d68f7	MGTT	2976f9fa-734c-474e-8398-c5be0a4ea120	D9A36E69 [DM.Pipeline.Common.TracingTelemetryService] Event: FireActivityCompletedWithFailureEvent (duration=3, err=CommunicationException, rootcauseErrorEventId=0)