Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Dataflow record deletion

Hi, Just to start off, I'm not keen on doing physical deletes but we have a need to carry out ad-hoc deletion of individual records after they have been loaded into a dataflow and I'm looking for a ...
Dataflow
lbendlin's avatar
lbendlin
Super User
2 years ago

Neither dataflows nor datasets offer CDC yet, and Microsoft have so far been unreceptive to our pleas.  You need to flush and fill the entire partition.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Thanks, that's interesting to know. Wouldn't be quite as bad if it was possible to schedule a "Full" load once in a while (eg. overnight). 

  • lbendlin's avatar
    lbendlin
    Super User
    2 years ago

    You have full control over the partition refresh on the dataset side (it behaves just like OLAP cube partitions) but to my knowledge there is no such option for dataflows. To force a full load you have to make meta data changes (ie mess with the Power Query).

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      Thanks again. I remember seeing a "Guy in the Cube" video about refreshing dataset partitions but unfortunately we're not at the stage where were building new reports from datasets yet. Dataflows have proved to be very useful to us except for this issue. Thanks for the confirmation that you have to "tweak" the Power Query to force a full load as thats been my experience so far. Its almost that once the data is checked in they don't want it to leave - aka "Hotel Califormia" ğŸ˜€

      • lbendlin's avatar
        lbendlin
        Super User
        2 years ago

        The only reason I can see for dataflows is to shield you (the developer) from slow data sources.  If your data source are not slow, don't use dataflows.