Refresh failing in table that uses incremental refresh after removing columns

[Anonymous]

My dataset has a table with data that comes from an SQL Server view. This table is configured to be incrementally refreshed. I recently changed the SQL Server view and removed a couple of columns from it. From the moment I did this change, the following error started to be displayed every time the scheduled refresh runs:

The '<pii>Column</pii>' column does not exist in the rowset.

In Power BI Desktop, I opened the PBIX file that gave origin to this dataset, refreshed it successfully and could see that the columns that were removed are no longer being shown in the table. I then published this updated dataset to Power BI Service but the error persists. I could also note a different behavior after doing this publish - the dataset no longer refreshes automatically immediately after the publish.

Is Power BI Service still referring to the old metadata (because the table uses incremental refresh) and therefore raising the error because it expects that the removed columns exist? If that's the case, how can I make Power BI Service understand that those columns no longer exist? Why is the dataset not refreshing immediately after being published?

[v-janeyg-msft]

Hi, @Anonymous 

Once the dataset is published and refreshed, if a change needs to be made to the model or reports, it needs to be republished from Power BI Desktop to the service. Your approach is correct.

I don't understand your doubts, the dataset won't refresh automatically. You need to set manual refresh or schedule refresh. Why do you think the dataset will refresh automatically?

In the service, refresh the dataset. The first refresh will load both new and updated data in the refresh period as well historical data for the entire store period. Depending on the amount of data, this can take quite a long time. Subsequent refreshes, either manual or scheduled are typically much faster because the incremental refresh policy is applied and only data for the period specified in the refresh policy setting is refreshed.

Configure incremental refresh and real-time data for Power BI datasets - Power BI | Microsoft Docs

Did I answer your question ? Please mark my reply as solution. Thank you very much.
If not, please feel free to ask me.

Best Regards,
Community Support Team _ Janey

[Anonymous]

From what I could understand in the meanwhile, it seems that the dataset is automatically refreshed immediately after being published if it has scheduled refreshed configured.

[v-janeyg-msft]

@Anonymous  

It also depends on how your schedule refresh is set. If it's time to refresh, it will refresh on time. There is no such thing as refreshing immediately after setting the schedule refresh.

[Anonymous]

That's not what I've been observing. I have a dataset that is scheduled to refresh weekly. Every time after I publish a new version of this dataset, it immediately and automatically starts a refresh.

I have another dataset without any scheduled refresh configured. It doesn't start an automatic refresh after a new version of the dataset is published.

[v-janeyg-msft]

Hi, @Anonymous 

It is indeed written in the document, and the republish report will be refreshed.  But I am not sure whether the logic of incremental refresh is the same as normal. Because the initial refresh of the incremental refresh and the data refreshed by the subsequent refresh are different.

Configuring incremental refresh and real-time data 

Republish or replace a dataset published from Power BI Desktop 

Can you provide a refresh history of your incremental refresh report?

[Anonymous]

The problem was in another table where I changed the name of a column. I refreshed that table in Power BI Desktop and now the dataset is being successfully refreshed in Power BI Service.

I still don't understand why isn't my dataset automatically refreshing immediately after I publish it - any ideas?