Forum Discussion
Dataflow and Snapshot
Hello,
I have a Power BI workspace and I set the Storage to my Azure Data Lake.
I created a Datafow and as expeced the Data are saved in my ADLS in CDM format.
What is strange, at least for me, is that every time I refresh the dataflow a new CSV file is created and the previous file
is not overwritten. Essentially a snapshot is created and added to the folder. Is there a way i can limit the number of snapshots created or my ADLS will grow to infinite size.
Thank you so much
4 Replies
Hi silcambro
Could you tell me what kind of datasource did you use to build your dataflow?
I think your issue may be that when you refresh your dataflow, your file in Azure Data Lake will create a new file instead of overwritting the previous file.
Does the new file have the same name as the previous one?
Please show me a screenshot of your issue.
You may try to delete the previous, and check whether the dataflow will work well.
The blogs as below may help you.
Blog: Dataflows in Power BI
Best Regards,
Rico Zhou
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hello,
thank you so much for your reply.
My source are CSV files in Azure Data Lake.
And as you Know when Data flow Runs it will generate a CDM Folder,
those are the file in one of the CDM Snapshot Folders after three refreshes:
orderheader.csv@snapshot=2020-12-04T22:07:00.6513096Z
orderheader.csv@snapshot=2020-12-09T18:34:44.7746893Z
orderheader.csv@snapshot=2020-12-24T21:25:08.5191499Z
and this is the Image
Dataflow Name : Orders
Query: orderheader
It is the normal behavior didn't set anything in particular.
Any ideas, if this is by design?
Thank you so much
This is by design so Microsoft can sell more storage space. There is no way to limit the space usage.
Azure Data Lake has lifecycle management options:
https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview
