Forum Discussion

VISRAI's avatar
VISRAI
New Member
3 months ago
Solved

Incremental Refresh Issue

Hi all, I recently applied incremental refresh in Power BI with SharePoint as the data source. After making these changes, the report refreshes successfully in Power BI Desktop. However, when I publi...
  • Poojara_D12's avatar
    3 months ago

    Hi VISRAI 

    This error pops up because of a mismatch between the schema of your historical data partitions and the fresh data coming in during a Service refresh. When you configure incremental refresh, Power BI splits your SharePoint dataset into separate, frozen time chunks based on your parameters. If a contributor added or removed the "ABC" column in a recent SharePoint file, your local Desktop preview might handle the discrepancy quietly, but the Power BI Service engine strictly enforces schema uniformity across all historical and new partitions during a scheduled refresh.

    To fix this, you need to go back into Power Query Desktop and ensure that the "ABC" column is explicitly handled so every table output has an identical structure. Before the step where you combine your SharePoint files, add a step to explicitly select or remove columns, or use a custom M code step like Table.SelectColumns(Source, {"RequiredColumn1", "RequiredColumn2"}, MissingField.UseNull) on the individual tables. This forces Power BI to insert a column of null values for "ABC" in the older tables where it doesn't exist, creating a matching schema across all files that won't crash when the Service tries to append the data into the incremental partitions.

     

  • Murtaza_Ghafoor's avatar
    3 months ago

    VISRAI 

    Why this is happening:

    "ABC" column exists in some tables but not all tables.

    This strongly indicates a schema inconsistency issue in share point.

    When you apply Incremental refresh it creates partitions in the PBI Service, Power BI demands that:

    All columns must have identical column structure.

    • identical data types.
    • identical transformation yields.
    • across all partitions/files/folders returned from the SharePoint.

    If one SharePoint file/table is missing the ABC or BAC column, PBI Service refresh can fail even if it works in Power BI Desktop.

     

    You can create exception in your power BI code to save guard this by adding following

     

    Power Query Code

    if Table.HasColumns(#"Previous Step", "ABC")

    then #"Previous Step"

    else Table.AddColumn(#"Previous Step", "ABC", each null)

     If this helps, ✓ Mark as Kudos | Help Others