Forum Discussion
On-premise Data Gateway Exception
We have publised a number of reports which use Direct Query (to SQL server) on a premium capacity. In the last couple of weeks there have been several occasions where all these report fail to load - All visualizations in the report fail to load with the message either "Unable to load data model" or "Database operation was cancelled due to a previous error".
The problem corrects itself after a couple of hours.
The On-Prem Data Gateway logs the following exception during this time.
Any idea what could be wrong?
DM.EnterpriseGateway Error: 0 : 2017-12-28T03:16:37.9703712Z DM.EnterpriseGateway 78bfbdc4-b932-46f9-8df4-c9ab2bd28f15 e339d9df-e1fa-4af2-a40d-2f5dd806dc27 MGSR e339d9df-e1fa-4af2-a40d-2f5dd806dc27 DB4431EE [DM.Pipeline.Common] Non-gateway exception encountered in activity scope: System.ServiceModel.CommunicationException: The communication object '{0}' is in the {1} state. Communication objects cannot be used for communication unless they are in the Opened state.
Server stack trace:
at Microsoft.ServiceBus.ClientWebSocketConnection.ThrowIfNotOpen()
at Microsoft.ServiceBus.ClientWebSocketConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.BufferedConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult.WriteCore()
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult..ctor(FramingDuplexSessionChannel channel, Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.OnBeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.OutputChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.SocketConnectionChannelListener`2.DuplexSessionChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Dispatcher.DuplexChannelBinder.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartSend(Boolean completedSynchronously)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, TimeSpan timeout, AsyncCallback callback, Object asyncState)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(ServiceChannel channel, ProxyOperationRuntime operation, Object[] ins, AsyncCallback callback, Object asyncState)
at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateTask(ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeTaskService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.PowerBI.DataMovement.Pipeline.InternalContracts.IGatewayTransferCallback.TransferCallbackAsync(Byte[] packet)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5_0.<<TransferImplAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
DM.EnterpriseGateway Error: 0 : 2017-12-28T03:16:37.9859976Z DM.EnterpriseGateway 78bfbdc4-b932-46f9-8df4-c9ab2bd28f15 e339d9df-e1fa-4af2-a40d-2f5dd806dc27 MGSR e339d9df-e1fa-4af2-a40d-2f5dd806dc27 C416A628 [DM.Pipeline.Common.TracingTelemetryService] Event: FireActivityCompletedWithFailureEvent (duration=37, err=CommunicationException, rootcauseErrorEventId=0)
DM.EnterpriseGateway Error: 0 : 2017-12-28T03:16:37.9859976Z DM.EnterpriseGateway 9c014220-0d4f-4948-98d2-12ca02cbb203 e339d9df-e1fa-4af2-a40d-2f5dd806dc27 MGRF e339d9df-e1fa-4af2-a40d-2f5dd806dc27 C9C615D0 [DM.Pipeline.Common] Non-gateway exception encountered in activity scope: System.ServiceModel.CommunicationException: The communication object '{0}' is in the {1} state. Communication objects cannot be used for communication unless they are in the Opened state.
Server stack trace:
at Microsoft.ServiceBus.ClientWebSocketConnection.ThrowIfNotOpen()
at Microsoft.ServiceBus.ClientWebSocketConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.BufferedConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult.WriteCore()
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult..ctor(FramingDuplexSessionChannel channel, Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.OnBeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.OutputChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.SocketConnectionChannelListener`2.DuplexSessionChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Dispatcher.DuplexChannelBinder.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartSend(Boolean completedSynchronously)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, TimeSpan timeout, AsyncCallback callback, Object asyncState)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(ServiceChannel channel, ProxyOperationRuntime operation, Object[] ins, AsyncCallback callback, Object asyncState)
at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateTask(ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeTaskService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.PowerBI.DataMovement.Pipeline.InternalContracts.IGatewayTransferCallback.TransferCallbackAsync(Byte[] packet)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5_0.<<TransferImplAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5_0.<<TransferImplAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
DM.EnterpriseGateway Error: 0 : 2017-12-28T03:16:37.9859976Z DM.EnterpriseGateway 9c014220-0d4f-4948-98d2-12ca02cbb203 e339d9df-e1fa-4af2-a40d-2f5dd806dc27 MGRF e339d9df-e1fa-4af2-a40d-2f5dd806dc27 F911D718 [DM.Pipeline.Common.TracingTelemetryService] Event: FireActivityCompletedWithFailureEvent (duration=358, err=CommunicationException, rootcauseErrorEventId=0)
DM.EnterpriseGateway Error: 0 : 2017-12-28T03:16:37.9859976Z DM.EnterpriseGateway c9286486-48ef-42d7-acbc-db6fcc8999d5 e339d9df-e1fa-4af2-a40d-2f5dd806dc27 MGST e339d9df-e1fa-4af2-a40d-2f5dd806dc27 0E92BE0E [DM.Pipeline.Common] Non-gateway exception encountered in activity scope: System.ServiceModel.CommunicationException: The communication object '{0}' is in the {1} state. Communication objects cannot be used for communication unless they are in the Opened state.
Server stack trace:
at Microsoft.ServiceBus.ClientWebSocketConnection.ThrowIfNotOpen()
at Microsoft.ServiceBus.ClientWebSocketConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.BufferedConnection.BeginWrite(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult.WriteCore()
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.SendAsyncResult..ctor(FramingDuplexSessionChannel channel, Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.FramingDuplexSessionChannel.OnBeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.Channels.OutputChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at Microsoft.ServiceBus.SocketConnectionChannelListener`2.DuplexSessionChannel.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Dispatcher.DuplexChannelBinder.BeginSend(Message message, TimeSpan timeout, AsyncCallback callback, Object state)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartSend(Boolean completedSynchronously)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, TimeSpan timeout, AsyncCallback callback, Object asyncState)
at System.ServiceModel.Channels.ServiceChannel.BeginCall(ServiceChannel channel, ProxyOperationRuntime operation, Object[] ins, AsyncCallback callback, Object asyncState)
at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateTask(ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeTaskService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.PowerBI.DataMovement.Pipeline.InternalContracts.IGatewayTransferCallback.TransferCallbackAsync(Byte[] packet)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5_0.<<TransferImplAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass5_0.<<TransferImplAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<TransferImplAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.ServiceModel.GatewayTransferService.<>c__DisplayClass4_0.<<TransferAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.PowerBI.DataMovement.Pipeline.Common.Diagnostics.PipelineTelemetryService.<ExecuteInActivity>d__7`1.MoveNext()
3 Replies
Hi Anonymous,
>>The communication object '{0}' is in the {1} state. Communication objects cannot be used for communication unless they are in the Opened state.
Based on your error message, it seems like the communication port has be occurred by other application or thread.
Did you try contact to administrator for ensure if any rules or new applications will occupied port?In addition, it seems like you used the proxy on your gateway, right?
If this is a case, please double check on your proxy setting to ensure it can works properly.Regards,
Xiaoxin Sheng
Hi Anonymous
Did you try contact to administrator for ensure if any rules or new applications will occupied port?
I am the administrator of the system on which the Gateway is running and I am pretty sure no rules have changed. We get this error almost everyday now for 15-20 mins - During this time all the visuals in the reports fail to load. The issue goes away automatically on its own.
In addition, it seems like you used the proxy on your gateway, right?
If this is a case, please double check on your proxy setting to ensure it can works properly.
This is the setting for the proxy - (I believe this is the default setting and we haven't changed it)
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>Any help or advice to fix this is appreciated.
Thanks
HI Anonymous,
>>We get this error almost everyday now for 15-20 mins - During this time all the visuals in the reports fail to load. The issue goes away automatically on its own.
It sounds like the issue more related to proxy server.
Do you check on that server if they had any operations will affect your gateway connect?
In addition, you can also submit a support ticket for further support.
Submit a support ticket
Regards,
Xiaoxin Sheng