Hi,
I have some Python code that works outside of Power BI with the full data set in a CSV file, the data being maybe 1,000 lines of text with two columns. When copying this over to a Power BI Visualisation it only works when a subset of the data is selected, if the full amount is selected it runs all the code (seen by writing to an external file at each stage) gets to the plot show command and after 5 minutes times out:
Error Message:
Scripts stop running after 5 minutes. To fix this, reduce your script's runtime, and try again.
Stack Trace:
Microsoft.PowerBI.ExploreServiceCommon.ScriptHandlerException: Scripts stop running after 5 minutes. To fix this, reduce your script's runtime, and try again. ---> Microsoft.PowerBI.Scripting.Python.Exceptions.PythonScriptTimeoutException: Python execution timeout.
The script execution was terminated, since it was running for more than 300000 miliseconds.
at Microsoft.PowerBI.Scripting.Python.PythonScriptWrapper.RunScript(String originalScript, Int32 timeoutMs)
at Microsoft.PowerBI.Client.Windows.Python.PythonScriptHandler.GenerateVisual(ScriptHandlerOptions options)
--- End of inner exception stack trace ---
at Microsoft.PowerBI.Client.Windows.Python.PythonScriptHandler.GenerateVisual(ScriptHandlerOptions options)
at Microsoft.PowerBI.ExploreServiceCommon.ScriptVisualCommandFlow.RunInternal(Stream dataShapeResultStream, QueryBindingDescriptor& bindingDescriptor)
at Microsoft.PowerBI.ExploreServiceCommon.ScriptVisualCommandFlow.Run(Stream dataShapeResultStream, QueryBindingDescriptor& bindingDescriptor)
at Microsoft.PowerBI.ExploreHost.SemanticQuery.ExecuteSemanticQueryFlow.TransformDataShapeResult(QueryCommand transformCommand, SemanticQueryDataShapeCommand command, Stream dataShapeResultStream, QueryBindingDescriptor& bindingDescriptor)
at Microsoft.PowerBI.ExploreHost.SemanticQuery.ExecuteSemanticQueryFlow.ExecuteDataQuery(IQueryResultDataWriter queryResultDataWriter, EngineDataModel engineDataModel, DataQuery query, Int32 queryId, ServiceErrorStatusCode& serviceErrorStatusCode)
at Microsoft.PowerBI.ExploreHost.SemanticQuery.ExecuteSemanticQueryFlow.ProcessAndWriteSemanticQueryCommands(IQueryResultsWriter queryResultsWriter, ExecuteSemanticQueryRequest request, EngineDataModel engineDataModel)
Invocation Stack Trace:
at Microsoft.Mashup.Host.Document.ExceptionExtensions.GetCurrentInvocationStackTrace()
at Microsoft.Mashup.Client.UI.Shared.StackTraceInfo..ctor(String exceptionStackTrace, String invocationStackTrace, String exceptionMessage)
at Microsoft.PowerBI.Client.Windows.ErrorHostService.GetErrorDetails(ShowErrorDialogArgs args)
at Microsoft.PowerBI.Client.Windows.ErrorHostService.<>c__DisplayClass2_0.<<ShowErrorDialog>b__0>d.MoveNext()
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
at Microsoft.PowerBI.Client.Windows.ErrorHostService.<>c__DisplayClass2_0.<ShowErrorDialog>b__0()
at Microsoft.Mashup.Host.Document.SynchronizationContextExtensions.<>c__DisplayClass1_0`1.<SendAndMarshalExceptions>b__0()
at Microsoft.Mashup.Host.Document.SynchronizationContextExtensions.<>c__DisplayClass0_1.<SendAndMarshalExceptions>b__0(Object null)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Delegate.DynamicInvokeImpl(Object[] args)
at System.Windows.Forms.Control.InvokeMarshaledCallbackDo(ThreadMethodEntry tme)
at System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Forms.Control.InvokeMarshaledCallback(ThreadMethodEntry tme)
at System.Windows.Forms.Control.InvokeMarshaledCallbacks()
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Form.ShowDialog(IWin32Window owner)
at Microsoft.Mashup.Client.UI.Shared.WebDialogs.WebDialog.<>n__0(IWindowHandle owner)
at Microsoft.Mashup.Client.UI.Shared.WindowManager.ShowModal[T](T dialog, Func`1 showModalFunction)
at Microsoft.PowerBI.Client.Program.<>c__DisplayClass4_0.<Main>b__1()
at Microsoft.PowerBI.Client.Windows.IExceptionHandlerExtensions.<>c__DisplayClass3_0.<HandleExceptionsWithNestedTasks>b__0()
at Microsoft.Mashup.Host.Document.ExceptionHandlerExtensions.HandleExceptions(IExceptionHandler exceptionHandler, Action action)
at Microsoft.PowerBI.Client.Program.Main(String[] args)
OS Version:
Microsoft Windows NT 10.0.17134.0 (x64 en-US)
CLR Version:
4.7 or later [Release Number = 461808]
Peak Virtual Memory:
38.6 GB
Private Memory:
810 MB
Peak Working Set:
930 MB
IE Version:
11.706.17134.0
User ID:
bda703ce-90bd-4ff9-8413-cec8ec81c936
Workbook Package Info:
1* - en-GB, Query Groups: 0, fastCombine: Disabled, runBackgroundAnalysis: True.
Telemetry Enabled:
True
Model Default Mode:
Import
Debugging the code outside of PowerBI doesn't show any large amounts of memory being used and completes within a second on the full data set read from a CSV. The code:
# The following code to create a dataframe and remove duplicated rows is always executed and acts as a preamble for your script:
# dataset = pandas.DataFrame(osversion_name, id)
# dataset = dataset.drop_duplicates()
# Paste or type your script code here:
# The following code to create a dataframe and remove duplicated rows is always executed and acts as a preamble for your script:
import pandas as pd
import os
dataset = dataset.dropna()
dataset = dataset.groupby('osversion_name').count()[['id']].reset_index()
centOS_7 = dataset[dataset['osversion_name'].str.contains('CentOS') & (dataset['osversion_name'].str.contains(' 7.'))]
centOS_6 = dataset[dataset['osversion_name'].str.contains('CentOS') & (dataset['osversion_name'].str.contains(' 6.'))]
centOS_5 = dataset[dataset['osversion_name'].str.contains('CentOS') & (dataset['osversion_name'].str.contains(' 5.'))]
centOS_7['system_name']='centOS_7'
centOS_6['system_name']='centOS_6'
centOS_5['system_name']='centOS_5'
redhat_7 = dataset[dataset['osversion_name'].str.contains('Red Hat Enterprise') & (dataset['osversion_name'].str.contains(' 7.'))]
redhat_7['system_name']='redhat_7'
redhat_6 = dataset[dataset['osversion_name'].str.contains('Red Hat Enterprise') & (dataset['osversion_name'].str.contains(' 6.'))]
redhat_6['system_name']='redhat_6'
redhat_5 = dataset[dataset['osversion_name'].str.contains('Red Hat Enterprise') & (dataset['osversion_name'].str.contains(' 5.'))]
redhat_5['system_name']='redhat_5'
m_w_s_7 = dataset[dataset['osversion_name'].str.contains('Microsoft Windows ') & (dataset['osversion_name'].str.contains(' 7.'))]
m_w_s_7['system_name']='m_w_s_7'
m_w_s_2012 = dataset[dataset['osversion_name'].str.contains('Microsoft Windows Server 2012')]
m_w_s_2012['system_name']='m_w_s_2012'
m_w_s_2008 = dataset[dataset['osversion_name'].str.contains('Microsoft Windows Server 2008')]
m_w_s_2008['system_name']='m_w_s_2008'
ubuntu_10 = dataset[dataset['osversion_name'].str.contains('Ubuntu')& dataset['osversion_name'].str.contains(' 10')]
ubuntu_10['system_name'] = 'ubuntu_10'
ubuntu_14 = dataset[dataset['osversion_name'].str.contains('Ubuntu')& dataset['osversion_name'].str.contains(' 14')]
ubuntu_14['system_name'] = 'ubuntu_14'
debian = dataset[dataset['osversion_name'].str.contains('Debian')]
debian['system_name']='Debian'
windows_10 = dataset[dataset['osversion_name'].str.contains('Windows') & (dataset['osversion_name'].str.contains(' 10'))]
windows_10['system_name'] = 'windows_10'
frames = [centOS_7, centOS_6, centOS_5, redhat_7, redhat_6, redhat_5, m_w_s_7, m_w_s_2012, m_w_s_2008, ubuntu_14, ubuntu_10, windows_10, debian]
result = pd.concat(frames)
result_2 = result.set_index("system_name", drop = True)
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
system_count = result.system_name.unique()
total_id = []
for i in system_count:
total_id.append(result_2.loc[i]['id'].sum())
fig = plt.figure(figsize=(30,5))
plt.bar(system_count, total_id)
plt.show()
To confirm it successfully shows the full plot when run outside of Power BI and it shows the graph when a subset of the data is shown. Any ideas or suggestions appreciated.
