Forum Discussion

AnHell's avatar
AnHell
Icon for Helper I rankHelper I
10 months ago
Solved

Update method "allow delete" is not working in data flow

Hi,   I have a data flow that reads data from a source and writes it to a destination.   If the data in an existing record changes or a new record appears, the options in Setting -> Update method...
dataflow
Need Help
  • v-dineshya's avatar
    v-dineshya
    10 months ago

    Hi AnHell ,

    Thank you for the update.

     

    Please try below structure to your data flow.

     

    1. In Source 1, Your filtered view from Fabric warehouse returns 1 record. In Source 2, The Dataverse destination table with 10 records.

     

    2. Exists Transformation, Configure to find records in Dataverse (Source 2) that do not exist in Source 1. This identifies records that should be deleted.

     

    3. Alter Row Transformation, Use deleteIf(true()) for all rows coming from the Exists transformation.

     

    4. Sink Transformation, Connect to Dataverse. Please enable Allow delete and Allow upsert. Set the Alternate Key Name to the correct primary key field.

     

    Note: Make sure the key column is defined so ADF can match rows for deletion.

     

    5. Run in a Pipeline, Not Just Debug Mode. Deletes won’t execute in debug mode. You must run the data flow inside a pipeline using an Execute Data Flow activity.

     

    Please refer below links.

    Sink transformation in mapping data flow - Azure Data Factory & Azure Synapse | Microsoft Learn

    in dataflow the update method allow delete is not working - Microsoft Q&A

    Data Flow - "Delete If" setting in Alter Row - Microsoft Q&A

     

    I hope this information helps. Please do let us know if you have any further queries.

     

    Regards,

    Dinesh

     

v-dineshya's avatar
v-dineshya
Icon for Community Support rankCommunity Support
10 months ago

Hi AnHell ,

Thank you for the update.

 

Please try below structure to your data flow.

 

1. In Source 1, Your filtered view from Fabric warehouse returns 1 record. In Source 2, The Dataverse destination table with 10 records.

 

2. Exists Transformation, Configure to find records in Dataverse (Source 2) that do not exist in Source 1. This identifies records that should be deleted.

 

3. Alter Row Transformation, Use deleteIf(true()) for all rows coming from the Exists transformation.

 

4. Sink Transformation, Connect to Dataverse. Please enable Allow delete and Allow upsert. Set the Alternate Key Name to the correct primary key field.

 

Note: Make sure the key column is defined so ADF can match rows for deletion.

 

5. Run in a Pipeline, Not Just Debug Mode. Deletes won’t execute in debug mode. You must run the data flow inside a pipeline using an Execute Data Flow activity.

 

Please refer below links.

Sink transformation in mapping data flow - Azure Data Factory & Azure Synapse | Microsoft Learn

in dataflow the update method allow delete is not working - Microsoft Q&A

Data Flow - "Delete If" setting in Alter Row - Microsoft Q&A

 

I hope this information helps. Please do let us know if you have any further queries.

 

Regards,

Dinesh

 

AnHell's avatar
AnHell
Icon for Helper I rankHelper I
10 months ago

Hi, v-dineshya 

 

That was the problem! ... I wasn't using the exit transformation.

 

I finally fixed it by adding the sink as a source that I use to do a join and detect the null values that I need to delete. But with the exit transformation, I find it easier to implement.

 

Thank you very much for everything!