Forum Discussion

dmkblesser's avatar
dmkblesser
Advocate II
11 months ago

'Alm_InvalidRequest_PurgeRequired' - Dataset xxx changes will cause data deletion

Hi Community, 

 

I'm running into this issue while we tried to promote our semantic model using the Fabric CLI. The script we use is this one  Analysis-Services/pbidevmode/fabricps-pbip at master · microsoft/Analysis-Services from @RuiRomanoMS  

 

I saw another member ran into the similar issue and the post is  community https://community.fabric.microsoft.com/t5/Developer/Error-quot-LRO-API-Error-Alm-InvalidRequest-PurgeRequired/m-p/4651090.

RuiRomanoMS  responded in the comments and saying this could be a bug. I'd like to follow up with Microsoft to see if we have any updates on this.

In my case, the pipeline deployed 13 semantic models and 12 of them went through successfully. We made the same change to all 13 semantic models which is disable the time intellgence option Auto date/time.  A sample change: 

All semantic models are using import mode with incremental refresh and data has been loaded into all of them. Somehow it failed on one semantic model.

 

I want to understand this error message better. In what kind of scenarios, we would expect to see this error? Like the other post mentioned, are we expecting to see the error if we are introducing a new column into the semantic model? What about column name change or data type change? (I'm assuming data type change won't work). 

 

Appreciate any feedback or comments!

 

thanks

 

 

 

Need Help

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    11 months ago

    Hi dmkblesser ,

     

    That error.

    Alm_InvalidRequest_PurgeRequired – Dataset <name> changes will cause data deletion

    basically means the deployment API has detected a schema change that would force the dataset to be dropped and fully reprocessed. A “purge required” scenario usually comes up in cases like:

    • Column rename or removal (lineage breaks)
    • Column data type change (incompatible with stored data)
    • Incremental refresh / partition definition changes
    • Changing or removing default date hierarchies (like the Auto date/time variation block you showed)

    Adding new columns or measures is normally safe and won’t trigger it, but removing or altering existing ones almost always will.

    In your case, disabling Auto date/time removed the default hierarchy metadata. Fabric sometimes treats this as a destructive change, which is why you see the purge-required error. What’s odd is that it only hit one of your 13 models, even though the same change was applied to all. That usually points to something unique about that model for example, existing partitions or relationships referencing the date hierarchy, which makes Fabric treat the change as breaking.

    At this point you’ve got two paths.

    1. Allow the purge and force a full refresh after deployment (expected if the model really depends on that hierarchy).
    2. I’d recommend raising a support ticket with Microsoft so it’s officially logged and can help influence future improvements. You can do that here.
      https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket

    Regards,
    Akhil.

    • dmkblesser's avatar
      dmkblesser
      Advocate II
      11 months ago

      Hi Anonymous. Thank you for your quick response. 

       

      When you say allow the purge and force a full refresh after deployment, how do I force the purge? 

      In the previous post, Rui and other community members suggest to either run the fabric-cli (we cannot use this option since we cannot modify the pipeline at this point) or use SSMS to run a refresh clear command. 

       

      I tried to use the SSMS option but it still doesn't work. Below is what I've done, please let me know if this is the same refresh clear command as mentioned in the post:

      After I ran the process clear command, I can confirm that there is no data in the table but the deployment still failed to the same error message. (In this case, the model only contains one table) 

       

      thanks

  • Anonymous's avatar
    Anonymous
    Not applicable
    11 months ago

    Hi dmkblesser ,

    Thanks for sharing the follow-up. Running Process Clear in SSMS only wipes the table data, but it doesn’t actually signal to Fabric that it can drop/recreate the dataset. That’s why you still hit the same Alm_InvalidRequest_PurgeRequired error the deployment engine is blocking the metadata change itself, not just the data. Right now the only supported ways to allow this purge are either through the Fabric CLI with the --allow-purge flag, or via a pipeline configured to permit purge. Since you mentioned you can’t adjust the pipeline and CLI isn’t an option, the best next step is to raise a support ticket with Microsoft so they can confirm if this is the expected behavior or a bug (especially since 12 of your models went through successfully and 1 did not).
    I’d recommend raising a support ticket with Microsoft so it’s officially logged and can help influence future improvements. You can do that here.
    https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket

    Regards,
    Akhil.

  • Anonymous's avatar
    Anonymous
    Not applicable
    11 months ago

    Hi dmkblesser ,

    Did you get a chance to try raising a support ticket with Microsoft for this one? Since the purge is being blocked at the deployment engine level, support will be able to confirm whether it’s expected behavior or a product bug. That way, you’ll have an official case logged and it can also help influence future improvements.

    Regards,
    Akhil.