Forum Discussion
Incremental & Full refresh
- 1 year ago
Yes, it's possible to implement both Incremental and Full Refresh in a single semantic model — but it requires using a combination of parameters, Power Query logic, and orchestrated refresh control via tools like Azure Data Factory, Power Automate, or Tabular Editor + XMLA scripting.
Step 1: Set Up Parameters in Power BI Desktop:
- Create a Boolean parameter called IsFullRefresh.
- Use this parameter in Power Query to control how much data is pulled:
* Use this parameter in Power Query to control how much data is pulled:
* If IsFullRefresh = FALSE, use RangeStart and RangeEnd to perform an incremental refresh.
Step 2: Modify Power Query to Respect IsFullRefresh:
let Source = ..., FilteredRows = if IsFullRefresh then Source else Table.SelectRows(Source, each [Date] >= RangeStart and [Date] < RangeEnd) in FilteredRowsStep 3: Configure Incremental Refresh Policy in Power BI Service:
- After publishing, enable incremental refresh with RangeStart and RangeEnd.
Step 4: Use XMLA or API Automation to Toggle Full vs. Incremental:
- Use Tabular Editor, Azure Automation, or the REST API to:
- Update the value of IsFullRefresh parameter before each scheduled refresh (e.g., via Power Automate flow).
- Trigger full refresh (on weekends) by setting IsFullRefresh = TRUE and refreshing the dataset.
- Revert back to FALSE for weekdays (incremental only).
Notes:
* You'll need service principal or admin control to automate via API.* Be careful with data volume to avoid timeouts on full refreshes.
If this addresses your question, I’d appreciate it if you could mark it as the accepted solution. Thank you!
Yes, it's possible to implement both Incremental and Full Refresh in a single semantic model — but it requires using a combination of parameters, Power Query logic, and orchestrated refresh control via tools like Azure Data Factory, Power Automate, or Tabular Editor + XMLA scripting.
Step 1: Set Up Parameters in Power BI Desktop:
- Create a Boolean parameter called IsFullRefresh.
- Use this parameter in Power Query to control how much data is pulled:
* Use this parameter in Power Query to control how much data is pulled:
* If IsFullRefresh = FALSE, use RangeStart and RangeEnd to perform an incremental refresh.
Step 2: Modify Power Query to Respect IsFullRefresh:
let
Source = ...,
FilteredRows = if IsFullRefresh then Source else Table.SelectRows(Source, each [Date] >= RangeStart and [Date] < RangeEnd)
in
FilteredRowsStep 3: Configure Incremental Refresh Policy in Power BI Service:
- After publishing, enable incremental refresh with RangeStart and RangeEnd.
Step 4: Use XMLA or API Automation to Toggle Full vs. Incremental:
- Use Tabular Editor, Azure Automation, or the REST API to:
- Update the value of IsFullRefresh parameter before each scheduled refresh (e.g., via Power Automate flow).
- Trigger full refresh (on weekends) by setting IsFullRefresh = TRUE and refreshing the dataset.
- Revert back to FALSE for weekdays (incremental only).
Notes:
* You'll need service principal or admin control to automate via API.
* Be careful with data volume to avoid timeouts on full refreshes.
If this addresses your question, I’d appreciate it if you could mark it as the accepted solution. Thank you!