Forum Discussion

bubbledep's avatar
bubbledep
Helper I
1 year ago
Solved

Power Query Doesn't Load my 6GB / 200K Rows into Excel's View, Even With Data Loading in Power Query

Hello guys, how you all doing,   I developed an M code to get the price value based on some conditions, and it seems to work in Power Query but it takes forever to load in Excel's table view (and t...
  • BeaBF's avatar
    BeaBF
    1 year ago

    bubbledep Let's try with:

     

    let
    Source = Table.Combine({PRICES_A, PRICES_B, PRICES_C, PRICES_D, PRICES_E, PRICES_F, PRICES_G, PRICES_H, PRICES_I, PRICES_J, PRICES_K, PRICES_L, PRICES_M, PRICES_N, PRICES_O, PRICES_P}),

    // Sort by DATE in ascending order
    SortedTable = Table.Sort(Source, {{"DATE", Order.Ascending}}),

    // Group by all condition columns to ensure we find previous prices within the same group
    GroupedTable = Table.Group(SortedTable, {"columnCONDITION1", "columnCONDITION2", "columnCONDITION3"},
    {{"All Data", each _, type table [DATE=nullable date, PRICE=nullable number]}}),

    // Add a column to find previous DATE and PRICE
    AddPreviousPrice = Table.AddColumn(GroupedTable, "All Data", each
    let
    Grouped = [All Data],
    Sorted = Table.Sort(Grouped, {{"DATE", Order.Ascending}}),
    AddPrevious = Table.AddColumn(Sorted, "Prev Date", each try Sorted[DATE]{List.PositionOf(Sorted[DATE], [DATE])-1} otherwise null),
    AddPrevPrice = Table.AddColumn(AddPrevious, "Prev Price", each try Sorted[PRICE]{List.PositionOf(Sorted[DATE], [DATE])-1} otherwise null)
    in
    AddPrevPrice, type table [DATE=nullable date, PRICE=nullable number, Prev Date=nullable date, Prev Price=nullable number]
    ),

    // Expand back the grouped data
    ExpandedTable = Table.ExpandTableColumn(AddPreviousPrice, "All Data", {"DATE", "PRICE", "Prev Date", "Prev Price"}),

    // Rename columns for clarity
    RenamedColumns = Table.RenameColumns(ExpandedTable, {{"Prev Date", "DATE D-1"}, {"Prev Price", "PRICE D-1"}})

    in
    RenamedColumns

     

    BBF