Forum Discussion
Deployment pipeline - Update on table set for incremental refresh
Hello,
I am using Deployment pipeline on a dataset which is using incremental refresh. I understand the limitations which are posted in the docs:
Republishing a dataset that doesn't use incremental refresh, to replace a dataset that has incremental refresh enabled.
Renaming a table that has incremental refresh enabled.
Renaming non-calculated columns in a table with incremental refresh enabled.
We get below error whenever we try to deploy the using deployment pipeline while using incremental refresh and changing something in same fact:
We are ok with loosing data in destination. Is there an option to just overwrite the dataset in destination even if there is change in the fact with incremental refresh enabled? We can refresh complete data again.
Thanks,
Vishal
We have the option to choose to continue the deployment,
even if it means to drop the data.
See more info here.
17 Replies
Hi Anonymous,
we are working right now to enable this scenario exactly.
should be live within 1-2 months.
Just encountered this same issue. I can't republish as the changes are in a Tabular Editor .bim file, not in Power BI Desktop. Is this feature still in the works?
Thanks.
Any news related to this? We have just encountered the similar situation and we are eager to hear if this is now possible to solve.
We have the option to choose to continue the deployment,
even if it means to drop the data.
See more info here.
Ok, then there are some other issue. Because we have an situation that we limited the decimal numbers from 9 to 4 (column type from decimal numer to fixed decimal number) and tried to deploy the new version using deployment pipeline but faced the can't stat deployment -error message. What I understand we should get this continue deployment dialog but we just get the can't start deployment dialog and because of that we can't do the deployment at all.
Can you share the exact error message you get, as well as the tech details in the message? we can then take a look and better help out.
Great, thanks for the update!
As I understand it, if a table or column gets a breaking change, the entire target dataset will be emptied and need a full refresh, not just the affected table, correct?
Is there documentation that describes what counts as a breaking change?
Thanks again!
Since this is eventually an AS model, you can try and search in the AS documentation.
However, there are so many options and so many different situations that can cause that, that it might be impossible to fully document it.
Anonymous can you explain what exactly is the problem?
there are many threads here so its hard to understand what problem you mean.Also note that we do support incremental refresh, and also support deploying breaking changes by letting the user choose to continue the deployment and drop the data.
Absolutely, thanks for reaching out Nimrod_Shalit. I think what I'm looking for is how to enable what you say "letting the user choose to continue the deployment and drop the data" because I don't seem to have that option. The popup message I get doesn't have the option to proceed:
How do I bypass this error/warning and push the deployment through, knowing and understanding (and wanting) to rebuild all incremental partitions? Is there an admin setting or toggle I'm missing somewhere? For instance, its very common for us to add new columns to our incrementally refreshed fact tables, warranting a full reload in these instances.
Hi Anonymous
According to the docs about deployment pipeline, it seems this is by design to avoid a data loss. I haven't found any workaround to this.
You may post an idea about this feature and the development team will know about it.
Reference: https://powerbi.microsoft.com/en-us/blog/announcing-deployment-pipelines-general-availability-ga/
Best Regards,
Community Support Team _ Jing ZhangIt can't be true (that it's by design) because if I use SSMS and clear the data completely (ie. no data exists any longer in the table) the error is still given. I even try clearing (deleting) all data for all tables including the date tables and it still gives this same error (that the deployment could result in loss of data)!! It's absolutely illogical and absurd.
After a few seconds I've determined that the error message is inaccurate / a red herring - which is always frustrating for a user. The loss of data (even when there is none) is not an issue. It appears the way to solve it (at least for me), is to remove the Incremental Refresh config on the table and publish. That at least allows for a successful deployment and for the column name to change. Later, I guess we need to reapply the Incremental Refresh configs. All very frustrating for this month's release!
Obviously, the error message isn't necessary if devs allowed the renaming when no records exist in the tables.
