Clarifying Data Transformation Process in Power BI When Using Python Script in 'Get Data'

[HamidBee]

Hi All, 

I have a specific query regarding the use of Python scripts for data import in Power BI. My process involves using the 'Get Data' feature in Power BI, where I choose Python as the data source. I provide the file path and execute some data transformation steps directly within the Python script at this stage, before the data is imported into Power BI.

My question is: Where exactly do these data transformations take place? Are they processed within the Power BI environment, or do they occur externally, in a manner similar to how query folding works with SQL data sources?.

To clarify further, I am not performing these transformations within the Power BI's Power Query using Python. Instead, these transformations are part of the Python script used in the initial data retrieval stage ('Get Data'). I'm trying to understand if Power BI simply imports the transformed data as-is, or if it plays any role in the transformation process, especially in terms of efficiency and performance.

Thanks in advance.

[HamidBee]

Hi @lbendlin. Thank you for your reply. Just to make sure I've understood correctly. Let's assume the Power BI report has not yet been published but is currently being built. I use Python to apply transformation steps when using the Get Data method. Are you saying this workload is done by the Python environment outside of Power BI?.

Which kind of leads me to my next question. If I import a file via Python, apply Power Query M transformations and then apply some more Python transformations in Power Query. Would this still mean all of the workload is pushed outside of Power BI to the python environment?.

Thanks in advance. 

[lbendlin]

Are you asking "will it fold?"  ?  I would doubt that but don't know for sure.  You can test it and seee for yourself.

[lbendlin]

Any Python or R script or visual mandates the use of a Personal Gateway.  That already tells you where the processing is happening - on your PC, or on the PC where the Personal Gateway app is running.

In my opinion this disqualifies the use of Python or R for any ETL work.  Doesn't seem to stop people from trying though, including attempts at data write back.