Forum Discussion

JonathanR's avatar
JonathanR
New Member
1 year ago
Solved

Tracking changes to data when refreshing the data source

Hi All,   I have a dataset that is connected to a spreadsheet (via web link) and I plan to set up a weekly automatic refresh. The spreadsheet is where all the data is changed/new rows are created. ...
  • pankajnamekar25's avatar
    1 year ago

    Hello JonathanR 

    Power BI alone doesn’t retain past data after refresh. To compare week-on-week:

    Create a Historical Table in your data model

    This stores all past versions of your data with a SnapshotDate column.

    Automate appending data weekly

    Use Power Automate, Power BI Dataflow, or an intermediate tool (like Power Query in Excel/SharePoint) to append the current week's snapshot to a long-term historical table.

    Add a column like SnapshotDate = DateTime.LocalNow() during each refresh.

    Thanks,
     Pankaj Namekar | LinkedIn

    If this solution helps, please accept it and give a kudos (Like), it would be greatly appreciated.

  • Akash_Varuna's avatar
    1 year ago

    JonathanR To track changes in your dataset during weekly refreshes, save each refreshed dataset into a "snapshot" table to retain historical data. Use queries or calculated columns to compare current data with the previous snapshot, identifying new rows or changes in specific fields. Create visuals to highlight these differences, such as tables for new entries or charts for status changes. Automate the process with Power Query and scheduled refresh to maintain consistent tracking.

  • v-sathmakuri's avatar
    v-sathmakuri
    1 year ago

    Hi JonathanR ,

     

    Thank you for reaching out to Microsoft Fabric Community.

     

    Thank you Akash_Varuna  and pankajnamekar25  for the prompt response.

     

    Create a pipeline and a warehouse in Microsoft Fabric.

     

    Use the Copy Data activity having your spreadsheet as the source and the warehouse as the destination. Load the data into a temporary table within the warehouse.

     

    Then, use a stored procedure to handle changes tracking, insert new records with the current timestamp using GETDATE().

     

    If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it! 

     

    Thank you!!