Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
bigk
Helper II
Helper II

Adding new rows and adjusting original row data

Hello community

I would like to ask for a help to solve my following issue. This related to transaction export from one investment platform and importing those to portfolio tracking tool.

The broker has made strange and i would say stupid data export functionality. If the stock was split or reverse split, then original transaction is being modified too. For example, i buy 15 pcs of stock1 for 10USD and then there is a 1:10 split (1:10), the original transaction of exported files will be adjusted too and would be 1.5 pcs for 100 USD. In portfolio tracking i need to show original transaction purchase quantity and price. The export file does not have that info, it has only quantity and total value. So i would have to manually calculate original purchase data.

I would like to ask for a help to build a query for this. Below is sample original data. The balance and previous balance columns are made to calculate cash flow. Because cash flow is not affected for splits, it should be copied over.

The required functionality is as follows:

1) calculated original purchase quantity (Units) based on stock split data per Position ID. The Units count from last row for each Position ID should be multiplied by total reverse splits. Then this value should be added to the row with “Open Position” value in Type for each Position ID.
2) Two new rows should be added for each split transaction per Position ID. One transaction will sell the stock with previous quantity values and another will add new quantities with split adjusted. Other data for new transactions should be copied from the “Open Position” transaction
3) All consequtive splits should use quantities from previous split, not from the Open Position.
4) The original lines with “corp action: Split” in “Type” column can stay or be removed. I can remove them later too.


The price of stock can be calculated by dividing Amount with Units so this should not be a problem.

Because extra lines will be added, it means that index columns would have to be recalculated to make sure that cash flow is calculated properly. Additionally a reverse split logic should be added. I don’t know how it would look like, but perhaps Split details column will be reverse, like 5:1

Below are source data and expected output table.

Than you for the big help. I’ve spent 4 days on this already and could not find solution.

 

Here is source data
let

Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("zZJBa8MwDIX/ism5cyRbkmVft9sOHWQ7hR5G1kNpacra/8/sJF1L6dZeBgODXuyHvxfJbVuB1oC1A4cGKXlOoNWsmu+WW/PS71eHVb/N383r/PG5fmuesmawALkiWnXgPWY9bZGBiBapuNgGn6sqchSkctwc+m69P9oXs7bCeE4nuJ9eivKA+IYHDNbHItFSyDWCqAuO/FW44xr8BJdEmU834Ihimcc7HJ/QakjIsisW47NJBwlBUB2hXsVDqPMa8T55TFAiH8mm2/T75cdFgCA28kXv3bSnJipZJ1kKgcV4s/slgZYE3kAYEhRf13/uzHtXQiTT7Darw3kIgwnh9OcP503IExmlKFiJ97XgbwPceAFhen5c+Bzu5MuP/DKAYfTkLMg/4P82gMUX", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Type = _t, Details = _t, Amount = _t, #"Units / Contracts" = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Position ID" = _t, #"Asset type" = _t, NWA = _t]),
#"Replaced Value2" = Table.ReplaceValue(Source,"-",null,Replacer.ReplaceValue,{ "Amount", "Units / Contracts", "Realized Equity Change", "Realized Equity", "Balance", "Position ID", "Asset type"}),
#"Replaced Value1" = Table.ReplaceValue(#"Replaced Value2",".",",",Replacer.ReplaceText,{"Amount","Units / Contracts","Realized Equity Change","Realized Equity","Balance","NWA"}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Replaced Value1", "Details", Splitter.SplitTextByEachDelimiter({"/"}, QuoteStyle.Csv, false), {"Details.1", "Details.2"}),
#"Split Column by Delimiter1" = Table.SplitColumn(#"Split Column by Delimiter", "Details.2", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, false), {"Details.2.1", "Details.2.2"}),
// Duplicated date to keep old date as text for reference needs
#"Duplicated Column" = Table.DuplicateColumn(#"Split Column by Delimiter1", "Date", "Date - Copy"),
#"Changed Type" = Table.TransformColumnTypes(#"Duplicated Column",{{"Date", type datetime}, {"Type", type text}, {"Details.1", type text}, {"Amount", type number}, {"Units / Contracts", type number}, {"Realized Equity Change", type number}, {"Realized Equity", type number}, {"Balance", type number}, {"Position ID", type text}, {"Asset type", type text}, {"NWA", Int64.Type}}),
#"Sorted Rows" = Table.Sort(#"Changed Type",{{"Date", Order.Ascending}}),
#"Renamed Columns" = Table.RenameColumns(#"Sorted Rows",{{"Details.2.1", "Currency"}, {"Details.2.2", "Split details"}, {"Details.1", "Details"}}),
// Filter On Open 1 USD comminssion lines as those have impact on balance and cash flow. Those do not reduce cash flow. Like unnecesary trx.
#"Filtered On Open commissions" = Table.SelectRows(#"Renamed Columns", each [Details] <> "On Open"),
// Index is needed to calculate cash previous balance and cash flow. Works in pair with Index.1
#"Added Index" = Table.AddIndexColumn(#"Filtered On Open commissions", "Index", 0, 1, Int64.Type),
// Index1 is needed to calculate cash previous balance and cash flow. Works in pair with Index
#"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1, Int64.Type),
// Calculated balance of previous row
#"Merged Queries" = Table.NestedJoin(#"Added Index1", {"Index"}, #"Added Index1", {"Index.1"}, "Added Index1", JoinKind.LeftOuter),
#"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries", "Added Index1", {"Balance"}, {"Added Index1.Balance"}),
#"Renamed Columns2" = Table.RenameColumns(#"Expanded Added Index1",{{"Added Index1.Balance", "Previous blance"}, {"Date - Copy", "Date ID"}, {"Units / Contracts", "Units"}}),
#"Renamed Columns3" = Table.RenameColumns(#"Renamed Columns2",{{"Date", "Transaction date"}}),
#"Inserted Date" = Table.AddColumn(#"Renamed Columns3", "Date", each DateTime.Date([Transaction date]), type date),
// Added price calculation for stock split adjustments. In PBI there is a separate price calculation.
AddedPrice = Table.AddColumn(#"Inserted Date", "PricePQ", each [Amount]/[Units])
in
AddedPrice



And here is expected output:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("tZXJbqQwEIZfBfU5MVV2eeM2yyWapUciOUU5IMKhNQRaAy1l3n5sBzQYSCtpkRaNzVb1f9Tv4v5+l9/8uPv+6fZm//O6KMv21PTXXV/01VPlZk9Fcyrq5LEq6o49193z7moHhgEyDhwTpEzIDIw7uz9WTfKr7Q79oW3ccX67//LNjXf5V7dvTnXtBgluh8gMByHQx3J/AosMyV+WTIv/dxuD0iokf1Pet+XvLmRPAdN5dmKcW/8zdogaMkVqdw9XF9CindISvJdWOjRhRlSNmgkvkZCRniBbUIZrTmLKinbKOua2THMgAiQ+4eSR1stYuWQgBlaVkUtJb2VFVExKr4cBl4MuQ4qYDMqEu274BBxBKzSc0EyJuUxBpHMFpNSI6kcRSb0MFTRz20sikQnMwIsaKZOybrvq8TVYrZiVMyfz4ZyxxplRuZkiYGgj+HVH69RtcynudYK1yjlaDOh+pEj65ejGPy8S0CGfj162f45JUXr6LMmP9aFf0GOG48oaXkQoMgRXKgNM2Qh7tcKe1qTz7JN4oRFEMrfEDGRJXoV0q3jLVbsgHCeri3YdcLloPaP6YM7Pp79nMYHJrTm9MyNOz6g34OR6aG7S55P6HbZVq7b16zT0JuIMVDRZ70166MaRgElIT2kipVuSnnWuerWkC8rzVV2HXFY1fIg+mHXdvVNUZ1ipxbas0gjuGji50EPgl2Y2g334Bw==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Source.Name = _t, #"Transaction date" = _t, Type = _t, Details = _t, Currency = _t, #"Split details" = _t, Amount = _t, Units = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Previous blance" = _t, #"Position ID" = _t, #"Asset type" = _t, #"Date ID" = _t, PricePQ = _t, NWA = _t, Index = _t, Index.1 = _t, Date = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Source.Name", type text}, {"Transaction date", type datetime}, {"Type", type text}, {"Details", type text}, {"Currency", type text}, {"Split details", type text}, {"Amount", type text}, {"Units", type text}, {"Realized Equity Change", type text}, {"Realized Equity", type text}, {"Balance", type text}, {"Previous blance", type text}, {"Position ID", Int64.Type}, {"Asset type", type text}, {"Date ID", type datetime}, {"PricePQ", type text}, {"NWA", Int64.Type}, {"Index", Int64.Type}, {"Index.1", Int64.Type}, {"Date", type date}})
in
#"Changed Type"

1 ACCEPTED SOLUTION

I think this should work with the updated sample and requirements. Quick summary of changes:

  • Updated Position ID transformation in FixValTypes to handle -'s as blanks
  • Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
  • Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
  • Within Group, at newGen step (List.Generate):
    • Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
    • Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)

 

let
    Source = Original_Updated,
    FixValsTypes = Table.TransformColumns(
        Source,
        {
            {"Date", each DateTime.From(_, "en-IN"), type datetime},
            {"Amount", each Number.FromText(_, "en-US"), Currency.Type},
            {
                "Units / Contracts", 
                each if _ = "-" then null else Number.FromText(_, "en-US"), 
                type number
            },
            {"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type},
            {
                "Realized Equity", 
                each Number.FromText(Text.Remove(_, " "), "en-IN"), 
                Currency.Type
            },
            {"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type},
            {"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type},
            {"NWA", each Number.FromText(_, "en-US"), Currency.Type}
        }
    ),
    SplitDetails = Table.SplitColumn(
        FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}), 
        {"Details", "Currency", "Split details"}
    ),
    AddSplitValue = Table.AddColumn(
        SplitDetails,
        "Split value",
        each
            if [Split details] = null then
                null
            else
                [
                    split = Text.Split([Split details], ":"),
                    output = Number.FromText(split{0}) / Number.FromText(split{1})
                ][output],
        type number
    ),
    NewType = type table Type.ForRecord(
        Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue)))
            & [
                Previous Balance = [Type = Currency.Type, Optional = false]
            ],
        false
    ),
    PreGroupSort = Table.Sort(
        AddSplitValue, {
            {"Position ID", Order.Ascending}, 
            {"Date", Order.Ascending}
        }
    ),
    Group = Table.Group(
        PreGroupSort,
        {"Position ID"},
        {
            "Fix",
            each [
                group = _,
                noOpen = not List.Contains( group[Type], "Open Position"),
                rowOrig = group{[Type = "Open Position"]},
                unitOrig = rowOrig[#"Units / Contracts"],
                splitProduct = List.Product(
                    List.Transform(
                        Table.SelectRows(
                            group, 
                            each [Type] = "corp action: Split"
                        )[Split value], 
                        each 1 / _
                    )
                ),
                unitFixed = unitOrig * splitProduct,
                amountOrig = rowOrig[Amount],
                amountFix = Table.ReplaceValue(
                    group, 
                    each [Amount], 
                    each 
                        if List.Contains( {"Position closed", "corp action: Split"}, [Type]) 
                        then amountOrig 
                        else [Amount], 
                    Replacer.ReplaceValue, 
                    {"Amount"}
                ),
                rowCount = Table.RowCount(group),
                addPrevBalance = Table.Buffer(
                    Table.FromColumns(
                        Table.ToColumns(amountFix) & {
                            {null} & List.RemoveLastN(group[Balance])
                        }, 
                        NewType
                    )
                ),
                firstRecordFixed = Record.TransformFields(
                    Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}}
                ),
                newGen = List.Generate(
                    () => [i = 0, rows = {firstRecordFixed}, unit = unitFixed],
                    each [i] < rowCount,
                    each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in
                        [
                            i = [i] + 1,
                            rows = 
                                [
                                    splitSell = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Sell"},
                                            {"Units / Contracts", each currentUnit}
                                        }
                                    ),
                                    splitBuy = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Buy"}, 
                                            {"Units / Contracts", each unit}
                                        }
                                    ),
                                    dividend = Record.TransformFields(
                                        currentRow,
                                        {{"Units / Contracts", each currentUnit}}
                                    ),
                                    noChange = currentRow,
                                    output = 
                                        if currentRow[Type] = "corp action: Split" 
                                            then {splitSell, splitBuy}
                                        else if currentRow[Type] = "Dividend" 
                                            then {dividend}
                                        else {noChange}
                                ][output],
                            unit = [unit] * (currentRow[Split value] ?? 1)
                        ],
                    each [rows]
                ),
                comboRows = List.Combine(newGen),
                output = if noOpen then group else Table.FromRecords(comboRows, NewType)
            ][output],
            NewType
        },
        GroupKind.Local
    ),
    CombineGroups = Table.Combine(Group[Fix]),
    AddPricePQ = Table.AddColumn(
        CombineGroups, "PricePQ", 
        each [Amount] / [#"Units / Contracts"], 
        type number
    )
in
    AddPricePQ

 

MarkLaf_0-1752792879761.png

 

View solution in original post

11 REPLIES 11
MarkLaf
Super User
Super User

I ended up going off of the Source step of your source data as some of the transformation steps weren't working in my default locale.

 

So, given:

 

Original (as provided by OP)

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("zZJBa8MwDIX/ism5cyRbkmVft9sOHWQ7hR5G1kNpacra/8/sJF1L6dZeBgODXuyHvxfJbVuB1oC1A4cGKXlOoNWsmu+WW/PS71eHVb/N383r/PG5fmuesmawALkiWnXgPWY9bZGBiBapuNgGn6sqchSkctwc+m69P9oXs7bCeE4nuJ9eivKA+IYHDNbHItFSyDWCqAuO/FW44xr8BJdEmU834Ihimcc7HJ/QakjIsisW47NJBwlBUB2hXsVDqPMa8T55TFAiH8mm2/T75cdFgCA28kXv3bSnJipZJ1kKgcV4s/slgZYE3kAYEhRf13/uzHtXQiTT7Darw3kIgwnh9OcP503IExmlKFiJ97XgbwPceAFhen5c+Bzu5MuP/DKAYfTkLMg/4P82gMUX", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Type = _t, Details = _t, Amount = _t, #"Units / Contracts" = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Position ID" = _t, #"Asset type" = _t, NWA = _t])
in
    Source

 

MarkLaf_0-1752618186501.png

 

I think the following will provide what you want. You can see I did my own value/type transforms up front (except for Int64, explicitly declared the locale). Majority of the work occurs in the Group step. Inside the group step, a List.Generate transform is used to construct the sell/buy rows.

 

let
    Source = Original,
    FixValsTypes = Table.TransformColumns(
        Source,
        {
            {"Date", each DateTime.From(_, "en-IN"), type datetime},
            {"Amount", each Number.FromText(_, "en-US"), Currency.Type},
            {"Units / Contracts", each if _ = "-" then null else Number.FromText(_, "en-US"), type number},
            {"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type},
            {"Realized Equity", each Number.FromText(Text.Remove(_, " "), "en-IN"), Currency.Type},
            {"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type},
            {"Position ID", Int64.From, Int64.Type},
            {"NWA", each Number.FromText(_, "en-US"), Currency.Type}
        }
    ),
    SplitDetails = Table.SplitColumn(
        FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}), 
        {"Details", "Currency", "Split details"}
    ),
    AddSplitValue = Table.AddColumn(
        SplitDetails,
        "Split value",
        each
            if [Split details] = null then
                null
            else
                [
                    split = Text.Split([Split details], ":"),
                    output = Number.FromText(split{0}) / Number.FromText(split{1})
                ][output],
        type number
    ),
    NewType = type table Type.ForRecord(
        Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue)))
            & [
                Previous Balance = [Type = Currency.Type, Optional = false]
            ],
        false
    ),
    PreGroupSort = Table.Sort(AddSplitValue, {{"Position ID", Order.Ascending}, {"Date", Order.Ascending}}),
    Group = Table.Group(
        PreGroupSort,
        {"Position ID"},
        {
            "Fix",
            each [
                group = _,
                rowOrig = group{[Type = "Open Position"]},
                unitOrig = rowOrig[#"Units / Contracts"],
                splitProduct = List.Product(
                    List.Transform(
                        Table.SelectRows(group, each [Type] = "corp action: Split")[Split value], 
                        each 1 / _
                    )
                ),
                unitFixed = unitOrig * splitProduct,
                amountOrig = rowOrig[Amount],
                amountFix = Table.ReplaceValue(
                    group, each [Amount], amountOrig, Replacer.ReplaceValue, {"Amount"}
                ),
                rowCount = Table.RowCount(group),
                addPrevBalance = Table.Buffer(
                    Table.FromColumns(
                        Table.ToColumns(amountFix) & {{null} & List.RemoveLastN(group[Balance])}, 
                        NewType
                    )
                ),
                firstRecordFixed = Record.TransformFields(
                    Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}}
                ),
                newGen = List.Generate(
                    () => [i = 0, rows = {firstRecordFixed}, unit = unitFixed],
                    each [i] < rowCount,
                    each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in
                        [
                            i = [i] + 1,
                            rows = if currentRow[Type] = "corp action: Split" then
                                [
                                    splitSell = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Sell"},
                                            {"Units / Contracts", each currentUnit}
                                        }
                                    ),
                                    splitBuy = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Buy"}, 
                                            {"Units / Contracts", each unit}
                                        }
                                    ),
                                    splitOutput = {splitSell, splitBuy}
                                ][splitOutput]
                            else
                                {currentRow},
                            unit = [unit] * currentRow[Split value]
                        ],
                    each [rows]
                ),
                comboRows = List.Combine(newGen),
                output = Table.FromRecords(comboRows, NewType)
            ][output],
            NewType
        },
        GroupKind.Local
    ),
    CombineGroups = Table.Combine(Group[Fix]),
    AddPricePQ = Table.AddColumn(
        CombineGroups, "PricePQ", 
        each [Amount] / [#"Units / Contracts"], 
        type number
    )
in
    AddPricePQ

 

MarkLaf_1-1752618434279.png

 

Hello @MarkLaf and thanks to looking into this. From the quick look it works as expected. When applying the query to full data set, some new discoveries.

There are some transactions that do not have position ID. those are mostly balance deposits or withdrawals. Because of that, the step "Group" has null values in Position ID and query runs into error.
Other thing is that transactions may be of some other types like fees and dividends and for them, the no data (units or amounts) should be adjusted. those should have original values as in the source.

 

I've added some new lines to source data so you can simulate that on your side. If you could help me sort these out, i could test the whole data set and see if it works as expected.

Thanks again.

 

= Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("zZRLb9pAEMe/yopzsp6Z3dmHbxWkl1aiKs0J5YDAad1SjLATqd++s7Z5OURwqVQJacdm1r+Z/zzm8xGEDDAjIFRoc8M5hNHdaLotNupLVZdNWW3kefZtOv6UPc4mYjNoADkRdSAwBsXuX1kFETXa5MXaGzlDQI4Obfp71lTLX/Xe/eluPsJ4SrdwOz0dgVvEAe7RaxOTidp6OSO4QJ6suQgnzsD0cJdb4dsrcESnmbtvEB/RQVlnNVNyUUacQmuCdxjIYriIB5/Jr8Ob3GAOKeQ9WS3XVV2sBgF4pyMPtKf+XVAxWE1OTGdBY7yqfoogpAiMAt9GkPyW1W6rFssURK5m23XZnAahMEc4Zn5/KoJUpDNdAO3ibRL82wCudIDv248Tn/2NfPcuPxWgLb0lDe4/4N9aAJC7OXPXg5Nim7ow9RYkV/Xw+FWNd8WqbMaLXepJMkaz6/GHh7fG/WmEbcaUQeyBmCpuUrjT12K3Kb//aNRzUQxaHrQ95tmaLOMN7RSYoPGGOZcVx4kp6wly4hwTc1K+lqtiM5ww0mT3eXVmkNXku3EyxDqkEIgMIAfP8bKsrieSQsxJOjb5PYh+Spy36nNV13t1QHs3GtSRUerY7rRI2tlBhmeCgghKhxFioaV7H1Y/X+rmd7Fpen9ZiHhuehXYdVUSPSleqhbYDKFbUaKc9bnls2rtiueXVsHJolz/6W7a40d69TDqtv2c92/LNf44ORCf/gI=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Type = _t, Details = _t, Amount = _t, #"Units / Contracts" = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Position ID" = _t, #"Asset type" = _t, NWA = _t])

 

So, to confirm the updated reqs:

 

1) Any Position ID groups (including where Position ID is null) that do not have an Open Position row included can just be left as is?

Example: no transformations are needed on the below outlined rows (1-2 & 6) as they have no associated Open Position?

MarkLaf_2-1752721319685.png

 

2) For Position ID groups with an Open Position row:

a) ALL other rows should use the Open Position Amount?

Example: for the below 906... group, the Amount for ALL rows should match Open Position's, i.e. 50?

MarkLaf_1-1752720953590.png

 

b) I'm guessing the Units should be based on most recent previous Units, whether that is Open Position or a split buy?

Example: If the timestamp of the Edit Stop Loss below put it after the first 1:10 split, then the associated Units should match the updated Units (which would be on the Split Buy row that gets added)?

MarkLaf_0-1752720756488.png

 

c) Sort of covered already indirectly, but want to directly check, any row with a timestamp that is earlier than the Open Position row should also get assigned the Open Position row's (recalculated, if group has splits) Units? Same with Amount? Or should it stay as is?

Example: What should happen to Units and Amount of the Overnight fee row in the 906... group?MarkLaf_3-1752724070901.png

 

 

1) correct. But if the position has dividend, then the "Units" should be adjusted based on the previous transaction "Units" count. E.g. if dividend is paid after "Open Position", then "Units" count from that "Open Position" adjusted for total splits should be used. If Dividend was paid after split, then "Units" count from earlier spit should be taken as Units.

2a) No, only Open Position and Stock Splits should have the same amount as those are not changing the position worth. All other types of transactions should have their original amounts. However for dividends, "Units" should be adjusted as per point 1.

2b) Correct. But only for dividends and splits. Other transactions should have original units (usually they don't have any) and amounts.

2c) There should not be any other transaction with the same Position ID before the "Open Position" transaction. This is a sampling error from my side. I've generated new sample data which is more clear.

If you check Position ID 881596140, you will see that Position Closed has different amount than Position open and split. That is because it was closed with profit. "Position Closed" transactions should also not be anyhow changed and kept as is.

 

= Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("zVRNb9swDP0rQs6tTFKivm5D0tMGZFjWU9BDkLibtywOErfA/v0o2/lomjbZYcBOpGVK7+k9itPpAEIBWBAQKrTJcIIwuBmM1+VKfa63VVPVK/mefB0PPxb3k5HkDBpAImoMBMag5P2SVRBl2eYq1t5IDAE5OrT596Sp5z+3u/KHm+kA4zG6hevRcwjcQuzBPXptYk5RWy8xggvkyZqz4OALcD24SQYT5D3zerNWs3mGTmqyXlbNMQOFCeGAeVaDIDQs57vv4rsaEBdgehouWZHBXtAA0Wnm7gziA3RQ1lnNlEuUkaLQpuCdsLQY3lTBv1Jhh6zmy3pbLk4IeKcjn1yf+rWgYrCanKTOgsZ4UYDMIGQGRoFvGcDf+nB7LII0Rpe6ANrF6yT4twTeb0Ty/SvgjM/+Snz3Jn42oLXekgb3H+BfawDI3sTc9eCoXOcuzL0FuVTd3X9Rw025qJrhbJN7kozR7Hr4/cfr5PaYYXtjKiBmQFKA2XGT6Y6fy82q+va9UY9ledLyoO3hnm3KMmWgfQUmaLw8bkgmLWdMmZKQiBNmzFH1XC3K1ekLI012d68uDTIhffecDLEOcHm+uR6QFGIiadi8+U7kU1K8Vp/q7XYnDmjvBic2MoqN7UiLpJ09wXuhJ4ietH9BLGh534fFj6dt86tcNX29jGV8mXoV2HUmiZwUz5kFtkDozYJkfWqH6sGsTfn41Ao4mlXL391OezikFw+jbrvPeX/RrYc/", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Type = _t, Details = _t, Amount = _t, #"Units / Contracts" = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Position ID" = _t, #"Asset type" = _t, NWA = _t])

 

I think this should work with the updated sample and requirements. Quick summary of changes:

  • Updated Position ID transformation in FixValTypes to handle -'s as blanks
  • Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
  • Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
  • Within Group, at newGen step (List.Generate):
    • Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
    • Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)

 

let
    Source = Original_Updated,
    FixValsTypes = Table.TransformColumns(
        Source,
        {
            {"Date", each DateTime.From(_, "en-IN"), type datetime},
            {"Amount", each Number.FromText(_, "en-US"), Currency.Type},
            {
                "Units / Contracts", 
                each if _ = "-" then null else Number.FromText(_, "en-US"), 
                type number
            },
            {"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type},
            {
                "Realized Equity", 
                each Number.FromText(Text.Remove(_, " "), "en-IN"), 
                Currency.Type
            },
            {"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type},
            {"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type},
            {"NWA", each Number.FromText(_, "en-US"), Currency.Type}
        }
    ),
    SplitDetails = Table.SplitColumn(
        FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}), 
        {"Details", "Currency", "Split details"}
    ),
    AddSplitValue = Table.AddColumn(
        SplitDetails,
        "Split value",
        each
            if [Split details] = null then
                null
            else
                [
                    split = Text.Split([Split details], ":"),
                    output = Number.FromText(split{0}) / Number.FromText(split{1})
                ][output],
        type number
    ),
    NewType = type table Type.ForRecord(
        Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue)))
            & [
                Previous Balance = [Type = Currency.Type, Optional = false]
            ],
        false
    ),
    PreGroupSort = Table.Sort(
        AddSplitValue, {
            {"Position ID", Order.Ascending}, 
            {"Date", Order.Ascending}
        }
    ),
    Group = Table.Group(
        PreGroupSort,
        {"Position ID"},
        {
            "Fix",
            each [
                group = _,
                noOpen = not List.Contains( group[Type], "Open Position"),
                rowOrig = group{[Type = "Open Position"]},
                unitOrig = rowOrig[#"Units / Contracts"],
                splitProduct = List.Product(
                    List.Transform(
                        Table.SelectRows(
                            group, 
                            each [Type] = "corp action: Split"
                        )[Split value], 
                        each 1 / _
                    )
                ),
                unitFixed = unitOrig * splitProduct,
                amountOrig = rowOrig[Amount],
                amountFix = Table.ReplaceValue(
                    group, 
                    each [Amount], 
                    each 
                        if List.Contains( {"Position closed", "corp action: Split"}, [Type]) 
                        then amountOrig 
                        else [Amount], 
                    Replacer.ReplaceValue, 
                    {"Amount"}
                ),
                rowCount = Table.RowCount(group),
                addPrevBalance = Table.Buffer(
                    Table.FromColumns(
                        Table.ToColumns(amountFix) & {
                            {null} & List.RemoveLastN(group[Balance])
                        }, 
                        NewType
                    )
                ),
                firstRecordFixed = Record.TransformFields(
                    Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}}
                ),
                newGen = List.Generate(
                    () => [i = 0, rows = {firstRecordFixed}, unit = unitFixed],
                    each [i] < rowCount,
                    each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in
                        [
                            i = [i] + 1,
                            rows = 
                                [
                                    splitSell = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Sell"},
                                            {"Units / Contracts", each currentUnit}
                                        }
                                    ),
                                    splitBuy = Record.TransformFields(
                                        currentRow,
                                        {
                                            {"Type", each "Split Buy"}, 
                                            {"Units / Contracts", each unit}
                                        }
                                    ),
                                    dividend = Record.TransformFields(
                                        currentRow,
                                        {{"Units / Contracts", each currentUnit}}
                                    ),
                                    noChange = currentRow,
                                    output = 
                                        if currentRow[Type] = "corp action: Split" 
                                            then {splitSell, splitBuy}
                                        else if currentRow[Type] = "Dividend" 
                                            then {dividend}
                                        else {noChange}
                                ][output],
                            unit = [unit] * (currentRow[Split value] ?? 1)
                        ],
                    each [rows]
                ),
                comboRows = List.Combine(newGen),
                output = if noOpen then group else Table.FromRecords(comboRows, NewType)
            ][output],
            NewType
        },
        GroupKind.Local
    ),
    CombineGroups = Table.Combine(Group[Fix]),
    AddPricePQ = Table.AddColumn(
        CombineGroups, "PricePQ", 
        each [Amount] / [#"Units / Contracts"], 
        type number
    )
in
    AddPricePQ

 

MarkLaf_0-1752792879761.png

 

From the first look it seems that everything works as expected. Maybe some things will be spotted later, but as of now this case can be closed.

Thanks @MarkLaf for yur help! You've saved me twice already including my other earlier request for help. 

And thanks @grazitti_sapna for your effort too!

PQ has limitless possibilities, unfortunately there is not enough time to learn everything. It is great to have such good community!

grazitti_sapna
Super User
Super User

Hi @bigk,

 

Try below code:

 

let
// Step 1: Load source data (already processed to AddedPrice)
Source = AddedPrice,

// Step 2: Extract Split rows and compute split ratios
SplitRows = Table.SelectRows(Source, each Text.Contains([Type], "Split")),
WithRatios = Table.AddColumn(SplitRows, "SplitRatio", each
let
raw = Text.Trim([Split details]),
parts = Text.Split(raw, ":"),
numerator = try Number.FromText(parts{0}) otherwise null,
denominator = try Number.FromText(parts{1}) otherwise null
in
if numerator <> null and denominator <> null then numerator / denominator else null,
type number),

// Step 3: Aggregate cumulative split ratio per Position ID
GroupedRatios = Table.Group(WithRatios, {"Position ID"}, {
{"TotalSplitFactor", each List.Product(List.RemoveNulls([SplitRatio])), type number}
}),

// Step 4: Join back to source to adjust Open Position rows
JoinedToMain = Table.NestedJoin(Source, {"Position ID"}, GroupedRatios, {"Position ID"}, "SplitFactor", JoinKind.LeftOuter),
ExpandedFactors = Table.ExpandTableColumn(JoinedToMain, "SplitFactor", {"TotalSplitFactor"}),
AdjustedOpen = Table.AddColumn(ExpandedFactors, "AdjustedUnits", each
if [Type] = "Open Position" and [TotalSplitFactor] <> null then
[Units] * [TotalSplitFactor]
else [Units],
type number),

// Step 5: Create synthetic rows for each split row
GeneratedSplits = Table.AddColumn(WithRatios, "SyntheticRows", each
let
originalRow = _,
positionID = originalRow[Position ID],
splitRatio = originalRow[SplitRatio],
date = originalRow[Transaction date],
baseRows = Table.SelectRows(Source, each [Position ID] = positionID and [Transaction date] < date and [Type] = "Open Position"),
unitsBefore = try baseRows{0}[Units] otherwise null,
unitsAfter = try unitsBefore * splitRatio otherwise null,
commonFields = Record.RemoveFields(originalRow, {"Type", "Units", "Amount"})
in
if unitsBefore <> null and unitsAfter <> null then {
Record.Combine({commonFields, [Type="Synthetic Split - Reverse", Units=-unitsBefore, Amount=null]}),
Record.Combine({commonFields, [Type="Synthetic Split - Apply", Units=unitsAfter, Amount=null]})
} else {}
),
FlattenedSynthetic = Table.Combine(List.Combine(GeneratedSplits[SyntheticRows])),

// Step 6: Combine and sort all rows
CombinedAll = Table.Combine({AdjustedOpen, FlattenedSynthetic}),
SortedFinal = Table.Sort(CombinedAll, {{"Transaction date", Order.Ascending}}),

// Step 7: Re-indexing and balance tracking
Reindexed = Table.AddIndexColumn(SortedFinal, "Index", 0, 1, Int64.Type),
WithIndex1 = Table.AddIndexColumn(Reindexed, "Index.1", 1, 1, Int64.Type),
MergedPrevBalance = Table.NestedJoin(WithIndex1, {"Index"}, WithIndex1, {"Index.1"}, "PrevRow", JoinKind.LeftOuter),
ExpandedPrevBalance = Table.ExpandTableColumn(MergedPrevBalance, "PrevRow", {"Balance"}, {"Previous blance"})

in
ExpandedPrevBalance

 

🌟 I hope this solution helps you unlock your Power BI potential! If you found it helpful, click 'Mark as Solution' to guide others toward the answers they need.
💡 Love the effort? Drop the kudos! Your appreciation fuels community spirit and innovation.
🎖 As a proud SuperUser and Microsoft Partner, we’re here to empower your data journey and the Power BI Community at large.
🔗 Curious to explore more? [Discover here].
Let’s keep building smarter solutions together!

Hello @grazitti_sapna and thanks for looking into this. I have 2 remarks. Adjusted open units were wrongly calculated but after changing sign from miltiply to dividsion, the adjusted open units were ok. Not sure how this will affect next steps as i could not proceed further due to the error in FlattenedSynthetic stepwhich gives an an error

Expression.Error: We cannot convert a value of type Record to type Table.

Hi @bigk,

 

Replace this Part

 

if unitsBefore <> null and unitsAfter <> null then {
Record.Combine({commonFields, [Type="Synthetic Split - Reverse", Units=-unitsBefore, Amount=null]}),
Record.Combine({commonFields, [Type="Synthetic Split - Apply", Units=unitsAfter, Amount=null]})
} else {}

 

with 

 

if unitsBefore <> null and unitsAfter <> null then {
{
Record.Combine({commonFields, [Type="Synthetic Split - Reverse", Units=-unitsBefore, Amount=null]}),
Record.Combine({commonFields, [Type="Synthetic Split - Apply", Units=unitsAfter, Amount=null]})
}
} else
{}

 

 

Thanks, but the issue still exist. Expression.Error: We cannot convert a value of type List to type Table.
Here is my full query. I've replaced first row to reference my original query that ends with AddedPrice.

let

// Step 1: Load source data (already processed to AddedPrice)
Source = #"Transactions SIMULATION",

// Step 2: Extract Split rows and compute split ratios
SplitRows = Table.SelectRows(Source, each Text.Contains([Type], "Split")),
WithRatios = Table.AddColumn(SplitRows, "SplitRatio", each
let
raw = Text.Trim([Split details]),
parts = Text.Split(raw, ":"),
numerator = try Number.FromText(parts{0}) otherwise null,
denominator = try Number.FromText(parts{1}) otherwise null
in
if numerator <> null and denominator <> null then numerator / denominator else null,
type number),

// Step 3: Aggregate cumulative split ratio per Position ID
GroupedRatios = Table.Group(WithRatios, {"Position ID"}, {
{"TotalSplitFactor", each List.Product(List.RemoveNulls([SplitRatio])), type number}
}),

// Step 4: Join back to source to adjust Open Position rows
JoinedToMain = Table.NestedJoin(Source, {"Position ID"}, GroupedRatios, {"Position ID"}, "SplitFactor", JoinKind.LeftOuter),
ExpandedFactors = Table.ExpandTableColumn(JoinedToMain, "SplitFactor", {"TotalSplitFactor"}),
AdjustedOpen = Table.AddColumn(ExpandedFactors, "AdjustedUnits", each
if [Type] = "Open Position" and [TotalSplitFactor] <> null then
[Units] * [TotalSplitFactor]
else [Units],
type number),

// Step 5: Create synthetic rows for each split row
GeneratedSplits = Table.AddColumn(WithRatios, "SyntheticRows", each
let
originalRow = _,
positionID = originalRow[Position ID],
splitRatio = originalRow[SplitRatio],
date = originalRow[Transaction date],
baseRows = Table.SelectRows(Source, each [Position ID] = positionID and [Transaction date] < date and [Type] = "Open Position"),
unitsBefore = try baseRows{0}[Units] otherwise null,
unitsAfter = try unitsBefore * splitRatio otherwise null,
commonFields = Record.RemoveFields(originalRow, {"Type", "Units", "Amount"})
in

if unitsBefore <> null and unitsAfter <> null then {
{
Record.Combine({commonFields, [Type="Synthetic Split - Reverse", Units=-unitsBefore, Amount=null]}),
Record.Combine({commonFields, [Type="Synthetic Split - Apply", Units=unitsAfter, Amount=null]})
}
} else
{}
),
FlattenedSynthetic = Table.Combine(List.Combine(GeneratedSplits[SyntheticRows])),

// Step 6: Combine and sort all rows
CombinedAll = Table.Combine({AdjustedOpen, FlattenedSynthetic}),
SortedFinal = Table.Sort(CombinedAll, {{"Transaction date", Order.Ascending}}),

// Step 7: Re-indexing and balance tracking
Reindexed = Table.AddIndexColumn(SortedFinal, "Index", 0, 1, Int64.Type),
WithIndex1 = Table.AddIndexColumn(Reindexed, "Index.1", 1, 1, Int64.Type),
MergedPrevBalance = Table.NestedJoin(WithIndex1, {"Index"}, WithIndex1, {"Index.1"}, "PrevRow", JoinKind.LeftOuter),
ExpandedPrevBalance = Table.ExpandTableColumn(MergedPrevBalance, "PrevRow", {"Balance"}, {"Previous blance"})

in
ExpandedPrevBalance

With the help of ChatGPT i've managed to get it working by modifying "FlattenedSynthetic" step to this:

Table.Combine(
List.Transform(
GeneratedSplits[SyntheticRows],
each Table.FromRecords(_)
)
)

However the issue now is that units are wrongly calcualted to split positions. First of all, units sould be multiplied by split ratio and not divided. But still other problem exists.

 

The GeneratedSplits steps uses [Units] from the source data for new transactions. However, split transactions in the source do not have units. they are null. Also negative synthetic rows seems the replicate value of split rows, however they should adjust the previous row and not the current one. For example first negative slit should adjust the "Open Position" units. But the second negative split should adjust the first split new units and so on.

 

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

July PBI25 Carousel

Power BI Monthly Update - July 2025

Check out the July 2025 Power BI update to learn about new features.

Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.

Top Solution Authors