How to selective delete and selectively load data in Dataflows
I have a requirement to delete data selectively from dataflow in PBI Service and load the same data again. This is needed as the source system ( SAP BW/4HANA) runs correction as and when it finds data quality issues. The data volume in the source is close to 12 Billion rows and it is aggregated and pulled into dataflow in PBI Service. The number of rows in the dataflow table is around 1.5 billion which rules out the possibility to reload complete data.
Data fix pattern is very random and can happen for any period, product etc.
How do I achieve this without disturbing the daily incremental refresh?