Forum Discussion
Lakehouse with cut off date
How can I build a solution that saves data from excel files stored in Sharepoint files with cut off dates for updating?
case scenario:
Manager want budget data until 1.september. This version have to be saved.
New data update with budget data until 1.oktober. This version have to be saved.
I need to have acces this two version so I can see what have changed.
Or make a cut off for updating every month can also work
Hi Felpan ,
Yes, To achieve this, you can utilize Dataflows Gen2 in Microsoft Fabric to ingest each budget version separately, assign timestamps to track updates, and store the data in a Lakehouse.
1. Set up Dataflows Gen2 to connect to your Excel files in SharePoint and import new data monthly.
2. In Power Query, add a column to label each file with a cut off date, such as 1Sep or 1Oct, for version tracking.
3. Load the data into a Lakehouse using the append option to store each month's version without overwriting previous ones.
4. Use Power BI to easily analyze and compare budget changes over time.
For a more detailed explanation, please review the official Microsoft documentation linked Below
Create your first Microsoft Fabric dataflow - Microsoft Fabric | Microsoft Learn
Helpful Reference:
Copy sample data into Lakehouse and transform with dataflow - Microsoft Fabric | Microsoft Learn
Yugandhar
CST Member.
5 Replies
Hi Felpan ,
Thank you for engaging with the Microsoft Fabric Community. I understand the importance of storing and comparing different versions of budget data.
You can use Dataflows Gen2 to ingest each budget version separately while assigning timestamps to track updates. The data can then be stored in a Lakehouse, ensuring historical versions are maintained for comparison. Finally, a Power BI report can be built with filters to help users track budget changes across different months.
Regards,
Yugandhar.
So how would this look like? ingest each budget version separately while assigning timestamps to track updates
Hi Felpan ,
Yes, To achieve this, you can utilize Dataflows Gen2 in Microsoft Fabric to ingest each budget version separately, assign timestamps to track updates, and store the data in a Lakehouse.
1. Set up Dataflows Gen2 to connect to your Excel files in SharePoint and import new data monthly.
2. In Power Query, add a column to label each file with a cut off date, such as 1Sep or 1Oct, for version tracking.
3. Load the data into a Lakehouse using the append option to store each month's version without overwriting previous ones.
4. Use Power BI to easily analyze and compare budget changes over time.
For a more detailed explanation, please review the official Microsoft documentation linked Below
Create your first Microsoft Fabric dataflow - Microsoft Fabric | Microsoft Learn
Helpful Reference:
Copy sample data into Lakehouse and transform with dataflow - Microsoft Fabric | Microsoft Learn
Yugandhar
CST Member.
Hi Felpan ,
I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.
Thank You.
Thanks for your help. I did not like to appende data, So the solution is to create a snapshot with Datapipeline.
