Forum Discussion
Does data cleaning automatically happen monthly?
hi kowas2
Since you steps involve combining worksheets then removing the duplicates, Power BI will remove the duplicates when you refresh the semantic model. Just be aware though that that Power BI will keep the first row it sees for each distinct value which may not be necessarily ordered by month.
Hello kowas2,
In Microsoft Power BI, your data cleaning steps (like Remove Duplicates) are saved in Power Query and only run when you refresh.
- Add new file → nothing happens yet
- Click Refresh (or use scheduled refresh)
- Power BI recombines files and removes duplicates again
You don’t need to remove duplicates manually each time.
If you want it monthly without action:
- Set up Scheduled Refresh in the Service
Yes, the duplicate removal happens automatically on every refresh. The steps you built in Power Query — combining the folder files and removing duplicates — are saved as a reusable recipe that runs every time the data refreshes. You never have to redo them manually.