Forum Discussion

kelvinlazam's avatar
kelvinlazam
Regular Visitor
11 months ago
Solved

Retaining historical data

Hi,

 

I have a dashboard that is connected to an SQL server. The data is about current invetory stocks.

My problem is I need to create a line graph which shows the stocks availability from previous months to the current month to monitor which months did we fail to stock up enough inventory.

 

The data extracted from SQL every refresh (weekly) is the updated to the current stock therefore I always lose the data from previous weeks.

 

Is it possible to solve this without extracting the data manually into a spreadsheet weekly?

 

Thank you!

  • 1) Create an append-only snapshot in SQL Server (recommended)

    You currently have a table/view with current stock only (it keeps changing). We’ll add a new table that appends one row per item/warehouse per snapshot date. That becomes your history.

    a) Make the table

     

    -- Run once
    CREATE TABLE dbo.InventorySnapshot (
    SnapshotDate date NOT NULL,
    ItemID int NOT NULL,
    WarehouseID int NOT NULL,
    QtyOnHand decimal(18,2) NOT NULL,
    CONSTRAINT PK_InventorySnapshot PRIMARY KEY (SnapshotDate, ItemID, WarehouseID)
    );

     

    b) Make a stored procedure that captures “today”

    CREATE OR ALTER PROCEDURE dbo.CaptureInventorySnapshot
    AS
    BEGIN
    SET NOCOUNT ON;

    DECLARE @d date = CAST(GETDATE() AS date);

    INSERT INTO dbo.InventorySnapshot (SnapshotDate, ItemID, WarehouseID, QtyOnHand)
    SELECT
    @d,
    ci.ItemID,
    ci.WarehouseID,
    ci.QtyOnHand
    FROM dbo.CurrentInventory AS ci -- <-- replace with your source table or view
    WHERE NOT EXISTS (
    SELECT 1
    FROM dbo.InventorySnapshot s
    WHERE s.SnapshotDate = @d
    AND s.ItemID = ci.ItemID
    AND s.WarehouseID = ci.WarehouseID
    );
    END

     

    c) Schedule it

    * SQL Server Agent → create a job to EXEC dbo.CaptureInventorySnapshot on your chosen cadence:

    • Weekly (to mirror your refresh) and
    • An extra run on the last day of each month (e.g., 23:55) so you always have a month-end point.

    * No Agent? Use Windows Task Scheduler + sqlcmd to run the proc on a schedule.

    Result: your snapshot table grows over time (append-only). You now have history.

     

     

    2) Model the data in Power BI

    a) Bring in tables

    • InventorySnapshot (Import mode).
    • A proper Date table (covering the period you’ll analyze). Mark it as Date table.

    b) Relationships

    Relate Date[Date] → InventorySnapshot[SnapshotDate] (many-to-one, single, active).

     

    3) Measures for month-by-month line chart

    You want each month to show the stock level as of the last snapshot in that month (or the latest snapshot on/before the month end).

    You like MAXX(ALLSELECTED(...)), so I’ll lean into that pattern.

     

    SOH (as of last selected date) :=
    VAR SelMaxDate =
    MAXX(ALLSELECTED('Date'[Date]), 'Date'[Date])
    VAR LastSnapOnOrBeforeSel =
    CALCULATE(
    MAX(InventorySnapshot[SnapshotDate]),
    ALL(InventorySnapshot[SnapshotDate]),
    InventorySnapshot[SnapshotDate] <= SelMaxDate
    )
    RETURN
    CALCULATE(
    SUM(InventorySnapshot[QtyOnHand]),
    InventorySnapshot[SnapshotDate] = LastSnapOnOrBeforeSel
    )

     

    For a monthly line (clean month-end points):

     

    SOH (Month End) :=
    VAR EOM = EOMONTH( MAX('Date'[Date]), 0 )
    VAR StartOfMonth = DATE( YEAR(EOM), MONTH(EOM), 1 )
    VAR LastSnapInMonth =
    CALCULATE(
    MAX(InventorySnapshot[SnapshotDate]),
    FILTER(
    ALL(InventorySnapshot[SnapshotDate]),
    InventorySnapshot[SnapshotDate] >= StartOfMonth &&
    InventorySnapshot[SnapshotDate] <= EOM
    )
    )
    RETURN
    IF(
    NOT ISBLANK(LastSnapInMonth),
    CALCULATE(
    SUM(InventorySnapshot[QtyOnHand]),
    InventorySnapshot[SnapshotDate] = LastSnapInMonth
    )
    )

     

    Visual setup

    • Axis: Date[Month] (e.g., a Month Year column or the Date hierarchy at Month level).
    • Values: SOH (Month End).
    • Legend (optional): Item, Warehouse, or Category.

    This yields a smooth month-by-month line from the snapshots you’re capturing.

     

    4) Add “under-stock” detection (optional KPI)

    If you keep a target/min stock per item (e.g., Item[MinStock]):

     

    Understock (Month End) :=
    VAR Val = [SOH (Month End)]
    VAR MinStock = SELECTEDVALUE( Item[MinStock], 0 )
    RETURN IF( NOT ISBLANK(Val) && Val < MinStock, 1, 0 )

     

     

    5) Performance & housekeeping

    • Add an index on (ItemID, WarehouseID, SnapshotDate) if the table grows large.
    • In Power BI, enable Incremental Refresh for InventorySnapshot (e.g., keep last 36 months, refresh last 2 months).
    • If a month has no snapshot, the measure automatically picks the last snapshot before the selected date (from SOH (as of last selected date)), so your chart remains continuous.

     

    6) Can’t touch SQL Server? Two solid alternatives

    A) Power Automate (no manual work)

    • Schedule a flow: SQL Server (Get rows) → append to Excel table in OneDrive/SharePoint or to another SQL table (Azure SQL, etc.).
    • The flow runs weekly + month-end, creating the same append-only history.
    • Point Power BI to the archive (Excel/SQL) and reuse the DAX above.

    B) If you have a transactions table

    * If there’s a fact like InventoryMovements (Receipts/Issues with dates), you can compute stock position by cumulative sum up to month end:

    • Stock as of date = Opening + SUM(Receipts – Issues up to that date).

    * This avoids snapshots entirely. But if you only have a “current stock” table, snapshots are the right route.

     

    That’s it—you’ll preserve history automatically and get a clear month-to-month line of stock levels to spot where you were under-stocked.

     

     

    I hope it will help.

7 Replies

  • Shahid12523's avatar
    Shahid12523
    Community Champion

    Create a historical table in SQL
    Append weekly snapshots using a scheduled SQL job or Power Automate.

     


    Use that table in Power BI
    Build your line graph using SnapshotDate to track stock trends over time.
    No manual exports needed—just automate the inserts and visualize the history.

  • kelvinlazam 

    Power BI imports the same data is that is available in your sql server, it will not keep the historic of record which were there in the table last week. I would suggest you to talk to your data engineer and ask them to change the data load strategy of the table. That would be the best solution.

     

    If you want to do it in power BI, then I think you can use incremental refresh policy. Implement strategy should be Refresh last 1 week of data and keep last N number of years of data. 

     

    Keep in mind, when you make changes to your semantic model, it refreshes the whole dataset again and all your historic data will be removed. 

     

     

     

     

    Connect on LinkedIn

     

     

     








    Did I answer your question? Mark my post as a solution!
    If I helped you, click on the Thumbs Up to give Kudos.

    Proud to be a Super User!


  • Hi kelvinlazam 

     

    Power BI reads whatever is in the source at refresh time, overwriting the previous data. If you are on Fabric, you can create a dataflow that appends its output to a desntination (if you need a granular data) or use Power Automate to run a query against a dataset to extract a snapshot of the calculations (not necessarily raw data) as a CSV and store it in a SharePoint. This should be scheduled before the expected semantic model refresh. You will then need to connect to the files stored in SharePoint.

  • 1) Create an append-only snapshot in SQL Server (recommended)

    You currently have a table/view with current stock only (it keeps changing). We’ll add a new table that appends one row per item/warehouse per snapshot date. That becomes your history.

    a) Make the table

     

    -- Run once
    CREATE TABLE dbo.InventorySnapshot (
    SnapshotDate date NOT NULL,
    ItemID int NOT NULL,
    WarehouseID int NOT NULL,
    QtyOnHand decimal(18,2) NOT NULL,
    CONSTRAINT PK_InventorySnapshot PRIMARY KEY (SnapshotDate, ItemID, WarehouseID)
    );

     

    b) Make a stored procedure that captures “today”

    CREATE OR ALTER PROCEDURE dbo.CaptureInventorySnapshot
    AS
    BEGIN
    SET NOCOUNT ON;

    DECLARE @d date = CAST(GETDATE() AS date);

    INSERT INTO dbo.InventorySnapshot (SnapshotDate, ItemID, WarehouseID, QtyOnHand)
    SELECT
    @d,
    ci.ItemID,
    ci.WarehouseID,
    ci.QtyOnHand
    FROM dbo.CurrentInventory AS ci -- <-- replace with your source table or view
    WHERE NOT EXISTS (
    SELECT 1
    FROM dbo.InventorySnapshot s
    WHERE s.SnapshotDate = @d
    AND s.ItemID = ci.ItemID
    AND s.WarehouseID = ci.WarehouseID
    );
    END

     

    c) Schedule it

    * SQL Server Agent → create a job to EXEC dbo.CaptureInventorySnapshot on your chosen cadence:

    • Weekly (to mirror your refresh) and
    • An extra run on the last day of each month (e.g., 23:55) so you always have a month-end point.

    * No Agent? Use Windows Task Scheduler + sqlcmd to run the proc on a schedule.

    Result: your snapshot table grows over time (append-only). You now have history.

     

     

    2) Model the data in Power BI

    a) Bring in tables

    • InventorySnapshot (Import mode).
    • A proper Date table (covering the period you’ll analyze). Mark it as Date table.

    b) Relationships

    Relate Date[Date] → InventorySnapshot[SnapshotDate] (many-to-one, single, active).

     

    3) Measures for month-by-month line chart

    You want each month to show the stock level as of the last snapshot in that month (or the latest snapshot on/before the month end).

    You like MAXX(ALLSELECTED(...)), so I’ll lean into that pattern.

     

    SOH (as of last selected date) :=
    VAR SelMaxDate =
    MAXX(ALLSELECTED('Date'[Date]), 'Date'[Date])
    VAR LastSnapOnOrBeforeSel =
    CALCULATE(
    MAX(InventorySnapshot[SnapshotDate]),
    ALL(InventorySnapshot[SnapshotDate]),
    InventorySnapshot[SnapshotDate] <= SelMaxDate
    )
    RETURN
    CALCULATE(
    SUM(InventorySnapshot[QtyOnHand]),
    InventorySnapshot[SnapshotDate] = LastSnapOnOrBeforeSel
    )

     

    For a monthly line (clean month-end points):

     

    SOH (Month End) :=
    VAR EOM = EOMONTH( MAX('Date'[Date]), 0 )
    VAR StartOfMonth = DATE( YEAR(EOM), MONTH(EOM), 1 )
    VAR LastSnapInMonth =
    CALCULATE(
    MAX(InventorySnapshot[SnapshotDate]),
    FILTER(
    ALL(InventorySnapshot[SnapshotDate]),
    InventorySnapshot[SnapshotDate] >= StartOfMonth &&
    InventorySnapshot[SnapshotDate] <= EOM
    )
    )
    RETURN
    IF(
    NOT ISBLANK(LastSnapInMonth),
    CALCULATE(
    SUM(InventorySnapshot[QtyOnHand]),
    InventorySnapshot[SnapshotDate] = LastSnapInMonth
    )
    )

     

    Visual setup

    • Axis: Date[Month] (e.g., a Month Year column or the Date hierarchy at Month level).
    • Values: SOH (Month End).
    • Legend (optional): Item, Warehouse, or Category.

    This yields a smooth month-by-month line from the snapshots you’re capturing.

     

    4) Add “under-stock” detection (optional KPI)

    If you keep a target/min stock per item (e.g., Item[MinStock]):

     

    Understock (Month End) :=
    VAR Val = [SOH (Month End)]
    VAR MinStock = SELECTEDVALUE( Item[MinStock], 0 )
    RETURN IF( NOT ISBLANK(Val) && Val < MinStock, 1, 0 )

     

     

    5) Performance & housekeeping

    • Add an index on (ItemID, WarehouseID, SnapshotDate) if the table grows large.
    • In Power BI, enable Incremental Refresh for InventorySnapshot (e.g., keep last 36 months, refresh last 2 months).
    • If a month has no snapshot, the measure automatically picks the last snapshot before the selected date (from SOH (as of last selected date)), so your chart remains continuous.

     

    6) Can’t touch SQL Server? Two solid alternatives

    A) Power Automate (no manual work)

    • Schedule a flow: SQL Server (Get rows) → append to Excel table in OneDrive/SharePoint or to another SQL table (Azure SQL, etc.).
    • The flow runs weekly + month-end, creating the same append-only history.
    • Point Power BI to the archive (Excel/SQL) and reuse the DAX above.

    B) If you have a transactions table

    * If there’s a fact like InventoryMovements (Receipts/Issues with dates), you can compute stock position by cumulative sum up to month end:

    • Stock as of date = Opening + SUM(Receipts – Issues up to that date).

    * This avoids snapshots entirely. But if you only have a “current stock” table, snapshots are the right route.

     

    That’s it—you’ll preserve history automatically and get a clear month-to-month line of stock levels to spot where you were under-stocked.

     

     

    I hope it will help.

  • Hi kelvinlazam,

    We haven’t heard back from you in a while regarding your issue. let us know if your issue has been resolved or if you still require support.

     

    Thank you.