Forum Discussion
Adding new rows and adjusting original row data
- 1 year ago
I think this should work with the updated sample and requirements. Quick summary of changes:
-
Updated Position ID transformation in FixValTypes to handle -'s as blanks
-
Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
-
Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
-
Within Group, at newGen step (List.Generate):
-
Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
-
Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)
-
let Source = Original_Updated, FixValsTypes = Table.TransformColumns( Source, { {"Date", each DateTime.From(_, "en-IN"), type datetime}, {"Amount", each Number.FromText(_, "en-US"), Currency.Type}, { "Units / Contracts", each if _ = "-" then null else Number.FromText(_, "en-US"), type number }, {"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type}, { "Realized Equity", each Number.FromText(Text.Remove(_, " "), "en-IN"), Currency.Type }, {"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type}, {"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type}, {"NWA", each Number.FromText(_, "en-US"), Currency.Type} } ), SplitDetails = Table.SplitColumn( FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}), {"Details", "Currency", "Split details"} ), AddSplitValue = Table.AddColumn( SplitDetails, "Split value", each if [Split details] = null then null else [ split = Text.Split([Split details], ":"), output = Number.FromText(split{0}) / Number.FromText(split{1}) ][output], type number ), NewType = type table Type.ForRecord( Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue))) & [ Previous Balance = [Type = Currency.Type, Optional = false] ], false ), PreGroupSort = Table.Sort( AddSplitValue, { {"Position ID", Order.Ascending}, {"Date", Order.Ascending} } ), Group = Table.Group( PreGroupSort, {"Position ID"}, { "Fix", each [ group = _, noOpen = not List.Contains( group[Type], "Open Position"), rowOrig = group{[Type = "Open Position"]}, unitOrig = rowOrig[#"Units / Contracts"], splitProduct = List.Product( List.Transform( Table.SelectRows( group, each [Type] = "corp action: Split" )[Split value], each 1 / _ ) ), unitFixed = unitOrig * splitProduct, amountOrig = rowOrig[Amount], amountFix = Table.ReplaceValue( group, each [Amount], each if List.Contains( {"Position closed", "corp action: Split"}, [Type]) then amountOrig else [Amount], Replacer.ReplaceValue, {"Amount"} ), rowCount = Table.RowCount(group), addPrevBalance = Table.Buffer( Table.FromColumns( Table.ToColumns(amountFix) & { {null} & List.RemoveLastN(group[Balance]) }, NewType ) ), firstRecordFixed = Record.TransformFields( Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}} ), newGen = List.Generate( () => [i = 0, rows = {firstRecordFixed}, unit = unitFixed], each [i] < rowCount, each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in [ i = [i] + 1, rows = [ splitSell = Record.TransformFields( currentRow, { {"Type", each "Split Sell"}, {"Units / Contracts", each currentUnit} } ), splitBuy = Record.TransformFields( currentRow, { {"Type", each "Split Buy"}, {"Units / Contracts", each unit} } ), dividend = Record.TransformFields( currentRow, {{"Units / Contracts", each currentUnit}} ), noChange = currentRow, output = if currentRow[Type] = "corp action: Split" then {splitSell, splitBuy} else if currentRow[Type] = "Dividend" then {dividend} else {noChange} ][output], unit = [unit] * (currentRow[Split value] ?? 1) ], each [rows] ), comboRows = List.Combine(newGen), output = if noOpen then group else Table.FromRecords(comboRows, NewType) ][output], NewType }, GroupKind.Local ), CombineGroups = Table.Combine(Group[Fix]), AddPricePQ = Table.AddColumn( CombineGroups, "PricePQ", each [Amount] / [#"Units / Contracts"], type number ) in AddPricePQ -
1) correct. But if the position has dividend, then the "Units" should be adjusted based on the previous transaction "Units" count. E.g. if dividend is paid after "Open Position", then "Units" count from that "Open Position" adjusted for total splits should be used. If Dividend was paid after split, then "Units" count from earlier spit should be taken as Units.
2a) No, only Open Position and Stock Splits should have the same amount as those are not changing the position worth. All other types of transactions should have their original amounts. However for dividends, "Units" should be adjusted as per point 1.
2b) Correct. But only for dividends and splits. Other transactions should have original units (usually they don't have any) and amounts.
2c) There should not be any other transaction with the same Position ID before the "Open Position" transaction. This is a sampling error from my side. I've generated new sample data which is more clear.
If you check Position ID 881596140, you will see that Position Closed has different amount than Position open and split. That is because it was closed with profit. "Position Closed" transactions should also not be anyhow changed and kept as is.
= Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("zVRNb9swDP0rQs6tTFKivm5D0tMGZFjWU9BDkLibtywOErfA/v0o2/lomjbZYcBOpGVK7+k9itPpAEIBWBAQKrTJcIIwuBmM1+VKfa63VVPVK/mefB0PPxb3k5HkDBpAImoMBMag5P2SVRBl2eYq1t5IDAE5OrT596Sp5z+3u/KHm+kA4zG6hevRcwjcQuzBPXptYk5RWy8xggvkyZqz4OALcD24SQYT5D3zerNWs3mGTmqyXlbNMQOFCeGAeVaDIDQs57vv4rsaEBdgehouWZHBXtAA0Wnm7gziA3RQ1lnNlEuUkaLQpuCdsLQY3lTBv1Jhh6zmy3pbLk4IeKcjn1yf+rWgYrCanKTOgsZ4UYDMIGQGRoFvGcDf+nB7LII0Rpe6ANrF6yT4twTeb0Ty/SvgjM/+Snz3Jn42oLXekgb3H+BfawDI3sTc9eCoXOcuzL0FuVTd3X9Rw025qJrhbJN7kozR7Hr4/cfr5PaYYXtjKiBmQFKA2XGT6Y6fy82q+va9UY9ledLyoO3hnm3KMmWgfQUmaLw8bkgmLWdMmZKQiBNmzFH1XC3K1ekLI012d68uDTIhffecDLEOcHm+uR6QFGIiadi8+U7kU1K8Vp/q7XYnDmjvBic2MoqN7UiLpJ09wXuhJ4ietH9BLGh534fFj6dt86tcNX29jGV8mXoV2HUmiZwUz5kFtkDozYJkfWqH6sGsTfn41Ao4mlXL391OezikFw+jbrvPeX/RrYc/", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Type = _t, Details = _t, Amount = _t, #"Units / Contracts" = _t, #"Realized Equity Change" = _t, #"Realized Equity" = _t, Balance = _t, #"Position ID" = _t, #"Asset type" = _t, NWA = _t])
I think this should work with the updated sample and requirements. Quick summary of changes:
-
Updated Position ID transformation in FixValTypes to handle -'s as blanks
-
Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
-
Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
-
Within Group, at newGen step (List.Generate):
-
Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
-
Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)
-
let
Source = Original_Updated,
FixValsTypes = Table.TransformColumns(
Source,
{
{"Date", each DateTime.From(_, "en-IN"), type datetime},
{"Amount", each Number.FromText(_, "en-US"), Currency.Type},
{
"Units / Contracts",
each if _ = "-" then null else Number.FromText(_, "en-US"),
type number
},
{"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type},
{
"Realized Equity",
each Number.FromText(Text.Remove(_, " "), "en-IN"),
Currency.Type
},
{"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type},
{"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type},
{"NWA", each Number.FromText(_, "en-US"), Currency.Type}
}
),
SplitDetails = Table.SplitColumn(
FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}),
{"Details", "Currency", "Split details"}
),
AddSplitValue = Table.AddColumn(
SplitDetails,
"Split value",
each
if [Split details] = null then
null
else
[
split = Text.Split([Split details], ":"),
output = Number.FromText(split{0}) / Number.FromText(split{1})
][output],
type number
),
NewType = type table Type.ForRecord(
Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue)))
& [
Previous Balance = [Type = Currency.Type, Optional = false]
],
false
),
PreGroupSort = Table.Sort(
AddSplitValue, {
{"Position ID", Order.Ascending},
{"Date", Order.Ascending}
}
),
Group = Table.Group(
PreGroupSort,
{"Position ID"},
{
"Fix",
each [
group = _,
noOpen = not List.Contains( group[Type], "Open Position"),
rowOrig = group{[Type = "Open Position"]},
unitOrig = rowOrig[#"Units / Contracts"],
splitProduct = List.Product(
List.Transform(
Table.SelectRows(
group,
each [Type] = "corp action: Split"
)[Split value],
each 1 / _
)
),
unitFixed = unitOrig * splitProduct,
amountOrig = rowOrig[Amount],
amountFix = Table.ReplaceValue(
group,
each [Amount],
each
if List.Contains( {"Position closed", "corp action: Split"}, [Type])
then amountOrig
else [Amount],
Replacer.ReplaceValue,
{"Amount"}
),
rowCount = Table.RowCount(group),
addPrevBalance = Table.Buffer(
Table.FromColumns(
Table.ToColumns(amountFix) & {
{null} & List.RemoveLastN(group[Balance])
},
NewType
)
),
firstRecordFixed = Record.TransformFields(
Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}}
),
newGen = List.Generate(
() => [i = 0, rows = {firstRecordFixed}, unit = unitFixed],
each [i] < rowCount,
each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in
[
i = [i] + 1,
rows =
[
splitSell = Record.TransformFields(
currentRow,
{
{"Type", each "Split Sell"},
{"Units / Contracts", each currentUnit}
}
),
splitBuy = Record.TransformFields(
currentRow,
{
{"Type", each "Split Buy"},
{"Units / Contracts", each unit}
}
),
dividend = Record.TransformFields(
currentRow,
{{"Units / Contracts", each currentUnit}}
),
noChange = currentRow,
output =
if currentRow[Type] = "corp action: Split"
then {splitSell, splitBuy}
else if currentRow[Type] = "Dividend"
then {dividend}
else {noChange}
][output],
unit = [unit] * (currentRow[Split value] ?? 1)
],
each [rows]
),
comboRows = List.Combine(newGen),
output = if noOpen then group else Table.FromRecords(comboRows, NewType)
][output],
NewType
},
GroupKind.Local
),
CombineGroups = Table.Combine(Group[Fix]),
AddPricePQ = Table.AddColumn(
CombineGroups, "PricePQ",
each [Amount] / [#"Units / Contracts"],
type number
)
in
AddPricePQ