How do I delete items from feeder table without deleting from master in PQ editor?

[Lottie89]

Hello all,

I am developing a report that will refresh monthly from a csv file. Each month a new csv with the previous months data will feed into this. Therefore, the mechanism is to load the data into a feeder table, clean it, append the co te ts of the feeder table to the master then clear the rows from the feeder ready for the next refresh.

The problem I'm having is when I clear the feeder it removes the data from the master too.

I have seen some other responses that talk about referencing one of the applied steps. I have been able to 'extract previous' from the applied step before I clear the feeder table and can link this new query to the append no problem. However, I can't understand how this would be a repeatable mechanism? Surely I would need to manually complete this upon every refresh for it to work?

I figured this reporting mechanism must be super common so I would love to understand what best practice is in this situation.

Thanks in advance

[OktayPamuk80]

Hi,

I can confirm the answer from our super users. You're trying to process something, which Power BI can't do: persist data. However, you can create incremental refresh or you process each time everything each reload. As you have monthly data, this would be the better option, as I think that you won't have much of data amount issue, moreover a reloading time, which isn't that bad thinking of having monthly refreshes.

Another option would be, can you implement this in another source system like SQL and SSIS,  Azure SQL DB or Fabric?

Did I answer your question? Then please mark my post as the solution.

If I helped you, click on the Thumbs Up to give Kudos.

[danextian]

Power BI reads the data and nothing else. It doesn't store a historical copy. Any change to your data will reflect in the next refresh. If you removed some rows, those rows would be removed from the semantic model as well. If you want to keep all the rows, either the source has it or connect to a folder connecting csv files of the monthly data.

[lbendlin]

Power BI has no memory.  The described approach to use self referencing has no safety net whatsoever.  Either handle your storage needs in the upstream system or consider using a Fabric data lake.