The python scikit-learn method, SVM, fit causes the timeout (although the script works well in Jupyter notebook)

[kcwagi]

Maybe someone in this group can help me with the following problem:

Problem Statement:

• I use a Python script in PowerBI. The script imports sklearn and pandas.
• In a Jupyter notebook the scrips work perfectly in less than a minute.
• When I use the same script in PowerBi (Edit Queries -> Transform -> Run Python Script), the script runs well, until the sklearn fit method is called. The classifier.fit method causes the script to time out ("Script execution completed because it was running for more than 1800000 milliseconds. Error code-2147467259")
  Note: I tested the script line by line to find the error. The script runs well until you called the tuning method (details below).

Classifier:

Clf = Svm. SVC(C-1,0, cache_size 200, class_weight-None, coef0-0,0,decision_function_shape'ovr', Grade 3 gamma 'scale', Core 'rbf', max_iter-1, probability-False, random_state-None, reduction of the truth, Toll 0.001, verbose-False)

Line that causes the problem:

clf.fit (X_learn_scaled, y_learn_selected_range)

What's different in PowerBI notebooks versus PowerBI Jupyther notebooks and why does the script work well on Jupyter notebooks, but not in PowerBI?

Thanks a lot

Christoph

[v-yingjl]

Hi @kcwagi ,

If you've fixed the issue on your own please kindly share your solution. If the above posts help, please kindly mark it as a solution to help others find it more quickly. Thanks!

Best Regards,
Yingjie Li

[v-yingjl]

Hi @kcwagi ,

1. Difference between Power BI python and Jupyther notebooks:

   • The Power BI Python integration requires the installation of two Python packages:

     • Pandas. A software library for data manipulation and analysis. It offers data structures and operations for manipulating numerical tables and time series. Your imported data must be in a pandas data frame. A data frame is a two-dimensional data structure. For example, data is aligned in a tabular fashion in rows and columns.
     • Matplotlib. A plotting library for Python and its numerical mathematics extension NumPy. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits, such as Tkinter, wxPython, Qt, or GTK+.

   • The notebook extends the console-based approach to interactive computing in a qualitatively new direction, providing a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results. The Jupyter notebook combines two components:

     A web application: a browser-based tool for interactive authoring of documents which combine explanatory text, mathematics, computations and their rich media output.

     Notebook documents: a representation of all content visible in the web application, including inputs and outputs of the computations, explanatory text, mathematics, images, and rich media representations of objects.

   • You can refer these two official documents for more details:
     • Run Python scripts in Power BI Desktop 
     • The Jupyter Notebook 
2. Based on your error message, the resaon that cause this scene is the script is over 30 minutes. When preparing and running a Python script in Power BI Desktop, there are a few limitations:
   • Only pandas data frames are imported, so make sure the data you want to import to Power BI is represented in a data frame
   • Any Python script that runs longer than 30 minutes times out
   • Interactive calls in the Python script, such as waiting for user input, halts the script’s execution
   • When setting the working directory within the Python script, you must define a full path to the working directory, rather than a relative path
   • Nested tables are currently not supported

Best Regards,
Yingjie Li

If this post helps then please consider Accept it as the solution to help the other members find it more quickly.