Forum Discussion
Need Help: Incremental Refresh Issues with MySQL Stored Procedure and User Hierarchy Bridge Table
- 1 year ago
Hi sanu ,
Thank you for reaching out to the Microsoft Community Forum.
Hi vojtechsima , lbendlin , Thank you for your prompt responses.
Hi sanu , In addition to vojtechsima ,lbendlin responses, I am adding some more points.
1. Power BI Service may not override RangeStart and RangeEnd parameters correctly after publishing. This will cause only partial data being refreshed. Use dummy variables in reference queries to pass RangeStart and RangeEnd explicitly and check the query folding is enabled .
2. Please check whether the stored procedure supports datetime parameters and returns consistent schema. And avoid referencing the stored procedure output in other queries, it will reflect the refresh behavior.
3. If UserType changes, consider those changes in a separate audit table or Use a surrogate key to differentiate historical records from current ones.
4. Please check the bridge table is refreshed properly that sync with User_View. If the stored procedure logic is complex, consider that output into a physical table and refresh that table. Use Power BI Dataflows to stage the bridge table separately and apply incremental refresh.
Question: Would it be better not to apply incremental refresh on both User_View and the bridge table, and instead perform a full refresh every time?Solution: Not necessary. Incremental refresh offers performance and reliability, especially for large datasets. if your stored procedure or view logic is too complex, full refresh might be more stable.
I hope this information helps. Please do let us know if you have any further queries.
Regards,
Dinesh
Hey, sanu ,
at first glance, I wouldn't use Incremental refresh for a slowly changing dimension (or perhaps accidental SCD), or any source where you can get duplicates (that happens because of the archive). I don't know how big the view/table is, but generally, if you have couple of mil. rows, it should be definitely fine. So Incremental refresh on fact table only.
Alternatively, create a proper Slowly changing dimension type 2, where you have from, to fields and you can filter with parameters on that.