Forum Discussion
Incremental refresh with MySQL as the backend
Hi Power BI Community,
I've recently implemented incremental refresh in three Power BI reports that use MySQL as the backend data source.
Report Design & Setup:
- The tables configured for incremental refresh utilize native SQL queries embedded directly within the Power BI report. Some of these native queries are quite complex.
- These tables represent the largest volume of data in their respective models.
- I've configured the standard RangeStart and RangeEnd parameters for incremental refresh.
Incremental Refresh Testing Results (Power BI Service):
After publishing and running the initial refresh, subsequent incremental refreshes showed the following improvements compared to full refresh times:
- Report 1: Approximately 50% reduction in refresh time.
- Report 2: Approximately 15% reduction in refresh time.
- Report 3: Approximately 10% reduction in refresh time.
Some Context:
- I see some references suggesting MySQL's query folding capabilities with Power BI for incremental refresh might be limited or may not work reliably for all transformations. I see that RangeStart and RangeEnd filters must to be pushed to the source for this feaure to really make any difference in the amount of data PowerBI needs to process in the refresh process.
- Currently, I do not have the necessary access to trace queries directly on the MySQL database logs or within Analysis Services logs to definitively confirm if the incremental refresh date range filters are being folded into the native SQL queries.
My Question:
Given these results, particularly for Report 1 where I saw a significant (50%) improvement in refresh time:
Is it a reasonable assumption that for Report 1, query folding is likely happening correctly, and the incremental refresh filter (based on RangeStart and RangeEnd) is being successfully folded into the native SQL query executed by MySQL?
And for the other reports (Report 2 and Report 3) with much lower improvements, could this indicate that query folding for the incremental date range is perhaps not occurring, or only partially occurring, leading to Power BI potentially pulling more data than necessary before filtering?
I'm trying to understand if the refresh time improvement alone can be a strong indicator of successful query folding for the incremental refresh mechanism in this MySQL scenario.
Any insights or experiences with MySQL incremental refresh and query folding would be greatly appreciated!
Thanks!
Hi ankurarora1418,
Thank you for reaching out to Microsoft Fabric Community Forum.
Akash_Varuna pankajnamekar25 Thank you for your quick responses.
ankurarora1418 Thanks for the detailed explanation, you're on the right track!
The insights shared by Akash_Varuna pankajnamekar25 are valid and on point. Just to add to their answers:
Yes, the significant refresh time improvement for Report 1 likely means that query folding is working, and the RangeStart/RangeEnd filters are being pushed down to MySQL.
For Reports 2 and 3, the smaller improvements suggest that query folding might not be happening, or only partially, which causes Power BI to pull more data than necessary before filtering.
While refresh time is a good indicator, but it’s not a guarantee. To confirm folding, you’d need to:
- Use the "View Native Query" option in Power Query (if enabled or not),
- Check the MySQL logs (if you have access),
- Test with a simplified version of the query.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
6 Replies
Hello ankurarora1418
refere this article and video
Power BI Incremental Refresh: The Ultimate Guide 2023
Thanks,
Pankaj Namekar | LinkedInIf this solution helps, please accept it and give a kudos (Like), it would be greatly appreciated.
Hi ankurarora1418 For Report 1, the 50% refresh improvement suggests query folding is likely working, applying RangeStart and RangeEnd filters. Lower gains in Reports 2 and 3 hint at partial or no folding, causing excess data processing. Without logs, improvements suggest folding but aren’t conclusive. Simplify queries to boost performance.
Hi ankurarora1418,
Thank you for reaching out to Microsoft Fabric Community Forum.
Akash_Varuna pankajnamekar25 Thank you for your quick responses.
ankurarora1418 Thanks for the detailed explanation, you're on the right track!
The insights shared by Akash_Varuna pankajnamekar25 are valid and on point. Just to add to their answers:
Yes, the significant refresh time improvement for Report 1 likely means that query folding is working, and the RangeStart/RangeEnd filters are being pushed down to MySQL.
For Reports 2 and 3, the smaller improvements suggest that query folding might not be happening, or only partially, which causes Power BI to pull more data than necessary before filtering.
While refresh time is a good indicator, but it’s not a guarantee. To confirm folding, you’d need to:
- Use the "View Native Query" option in Power Query (if enabled or not),
- Check the MySQL logs (if you have access),
- Test with a simplified version of the query.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
Hi ankurarora1418 ,
Thank you for reaching out to Microsoft Fabric Community Forum.
Akash_Varuna pankajnamekar25 Thank you for your quick response.
We wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful.
Please don't forget to give a "Kudos" – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
Hi ankurarora1418 ,
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
If our response addressed, please mark it as Accept as solution and click Yes if you found it helpful
Please don't forget to give a "Kudos " – I’d truly appreciate it!
Regards,
B Manikanteswara Reddy
Hi ankurarora1418 ,
Thank you for reaching out to Microsoft Fabric Community Forum.
As we haven’t heard back from you, we will go ahead and close this ticket for now.
If you need any further assistance, please don’t hesitate to raise a new ticket, we’re always happy to help.
Our sincere apologies if there was any inconvenience caused.
Regards,
B Manikanteswara Reddy
