Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
Leroy
Frequent Visitor

Power BI Incremental Refresh with Python Script Data Source

Hello everyone!

I'm new to Power BI and I need some advice: at the company where I work, we deal with a large volume of data. To create a dataset for analysis in Power BI, we use a Python script that fetches data from SQL databases and transforms it. The data needs to be updated daily, which takes a lot of time, and I'm thinking about how to speed up the report refresh process.

 

I've been reading about Incremental Refresh and tried to implement the idea, but I couldn't get it to work. After adding parameters RangeStart and RangeEnd to filter dates in the required table, for some reason, the refresh won't happen because I supposedly didn't add the parameters (even though they are added).

 

Does anyone have any ideas on how to speed up report refreshes? Has anyone managed to configure Incremental Refresh for data sourced from a Python Script?

I would be very grateful for any advice) 

4 REPLIES 4
v-yilong-msft
Community Support
Community Support

Hi  @Leroy ,

Have you solved your problem? If so, can you share your solution here and mark the correct answer as a standard answer to help other members find it faster? Thank you very much for your kind cooperation!

 

 

Best Regards

Yilong Zhou

 

lbendlin
Super User
Super User

You can create an Incremental Refresh setup against the SQL database.  It is not very productive to try this against the Python script as the query will not fold and there will be no performance benefit.

Hi @lbendlin 
We cannot directly connect to SQL databases in Power BI. A Python script collects data from our databases into one dataframe for analysis, which we then place into Power BI for visualization.

As I understand it, if the data source is a Python script, there is no way to configure Incremental Refresh in Power BI at all, right?

That's not what I said.  You can still configure incremental Refresh but it won't give you any performance improvement.

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.