Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
HamidBee
Impactful Individual
Impactful Individual

Clarifying Data Transformation Process in Power BI When Using Python Script in 'Get Data'

Hi All, 

 

I have a specific query regarding the use of Python scripts for data import in Power BI. My process involves using the 'Get Data' feature in Power BI, where I choose Python as the data source. I provide the file path and execute some data transformation steps directly within the Python script at this stage, before the data is imported into Power BI.

 

My question is: Where exactly do these data transformations take place? Are they processed within the Power BI environment, or do they occur externally, in a manner similar to how query folding works with SQL data sources?.

 

To clarify further, I am not performing these transformations within the Power BI's Power Query using Python. Instead, these transformations are part of the Python script used in the initial data retrieval stage ('Get Data'). I'm trying to understand if Power BI simply imports the transformed data as-is, or if it plays any role in the transformation process, especially in terms of efficiency and performance.

 

Thanks in advance.

3 REPLIES 3
HamidBee
Impactful Individual
Impactful Individual

Hi @lbendlin. Thank you for your reply. Just to make sure I've understood correctly. Let's assume the Power BI report has not yet been published but is currently being built. I use Python to apply transformation steps when using the Get Data method. Are you saying this workload is done by the Python environment outside of Power BI?.

 

Which kind of leads me to my next question. If I import a file via Python, apply Power Query M transformations and then apply some more Python transformations in Power Query. Would this still mean all of the workload is pushed outside of Power BI to the python environment?.

Thanks in advance. 

 

Are you asking "will it fold?"  ?  I would doubt that but don't know for sure.  You can test it and seee for yourself.

lbendlin
Super User
Super User

Any Python or R script or visual mandates the use of a Personal Gateway.  That already tells you where the processing is happening - on your PC, or on the PC where the Personal Gateway app is running.

 

In my opinion this disqualifies the use of Python or R for any ETL work.  Doesn't seem to stop people from trying though, including attempts at data write back.  

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.