Forum Discussion
bigk
1 year agoHelper III
Adding new rows and adjusting original row data
Hello community I would like to ask for a help to solve my following issue. This related to transaction export from one investment platform and importing those to portfolio tracking tool. The broke...
- 1 year ago
I think this should work with the updated sample and requirements. Quick summary of changes:
-
Updated Position ID transformation in FixValTypes to handle -'s as blanks
-
Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
-
Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
-
Within Group, at newGen step (List.Generate):
-
Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
-
Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)
-
let Source = Original_Updated, FixValsTypes = Table.TransformColumns( Source, { {"Date", each DateTime.From(_, "en-IN"), type datetime}, {"Amount", each Number.FromText(_, "en-US"), Currency.Type}, { "Units / Contracts", each if _ = "-" then null else Number.FromText(_, "en-US"), type number }, {"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type}, { "Realized Equity", each Number.FromText(Text.Remove(_, " "), "en-IN"), Currency.Type }, {"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type}, {"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type}, {"NWA", each Number.FromText(_, "en-US"), Currency.Type} } ), SplitDetails = Table.SplitColumn( FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}), {"Details", "Currency", "Split details"} ), AddSplitValue = Table.AddColumn( SplitDetails, "Split value", each if [Split details] = null then null else [ split = Text.Split([Split details], ":"), output = Number.FromText(split{0}) / Number.FromText(split{1}) ][output], type number ), NewType = type table Type.ForRecord( Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue))) & [ Previous Balance = [Type = Currency.Type, Optional = false] ], false ), PreGroupSort = Table.Sort( AddSplitValue, { {"Position ID", Order.Ascending}, {"Date", Order.Ascending} } ), Group = Table.Group( PreGroupSort, {"Position ID"}, { "Fix", each [ group = _, noOpen = not List.Contains( group[Type], "Open Position"), rowOrig = group{[Type = "Open Position"]}, unitOrig = rowOrig[#"Units / Contracts"], splitProduct = List.Product( List.Transform( Table.SelectRows( group, each [Type] = "corp action: Split" )[Split value], each 1 / _ ) ), unitFixed = unitOrig * splitProduct, amountOrig = rowOrig[Amount], amountFix = Table.ReplaceValue( group, each [Amount], each if List.Contains( {"Position closed", "corp action: Split"}, [Type]) then amountOrig else [Amount], Replacer.ReplaceValue, {"Amount"} ), rowCount = Table.RowCount(group), addPrevBalance = Table.Buffer( Table.FromColumns( Table.ToColumns(amountFix) & { {null} & List.RemoveLastN(group[Balance]) }, NewType ) ), firstRecordFixed = Record.TransformFields( Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}} ), newGen = List.Generate( () => [i = 0, rows = {firstRecordFixed}, unit = unitFixed], each [i] < rowCount, each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in [ i = [i] + 1, rows = [ splitSell = Record.TransformFields( currentRow, { {"Type", each "Split Sell"}, {"Units / Contracts", each currentUnit} } ), splitBuy = Record.TransformFields( currentRow, { {"Type", each "Split Buy"}, {"Units / Contracts", each unit} } ), dividend = Record.TransformFields( currentRow, {{"Units / Contracts", each currentUnit}} ), noChange = currentRow, output = if currentRow[Type] = "corp action: Split" then {splitSell, splitBuy} else if currentRow[Type] = "Dividend" then {dividend} else {noChange} ][output], unit = [unit] * (currentRow[Split value] ?? 1) ], each [rows] ), comboRows = List.Combine(newGen), output = if noOpen then group else Table.FromRecords(comboRows, NewType) ][output], NewType }, GroupKind.Local ), CombineGroups = Table.Combine(Group[Fix]), AddPricePQ = Table.AddColumn( CombineGroups, "PricePQ", each [Amount] / [#"Units / Contracts"], type number ) in AddPricePQ -
MarkLaf
1 year agoSuper User
I think this should work with the updated sample and requirements. Quick summary of changes:
-
Updated Position ID transformation in FixValTypes to handle -'s as blanks
-
Within Group, defined noOpen check and used in output step to just return grouped rows as is if no Open Positions are found in group
-
Within Group, updated amountFixed step to only apply Open Position Amount to Types "Position closed" and "corp action: Split"
-
Within Group, at newGen step (List.Generate):
-
Updated rows iteration to apply current Units to dividends (also, reorganized conditional for clarity)
-
Updated unit iteration to have the multiplied split value default to 1 (so, keep unit as is) in case split value is null (otherwise, dividend or any other non-split transactions with null split value would null out the unit getting passed forward)
-
let
Source = Original_Updated,
FixValsTypes = Table.TransformColumns(
Source,
{
{"Date", each DateTime.From(_, "en-IN"), type datetime},
{"Amount", each Number.FromText(_, "en-US"), Currency.Type},
{
"Units / Contracts",
each if _ = "-" then null else Number.FromText(_, "en-US"),
type number
},
{"Realized Equity Change", each Number.FromText(_, "en-US"), Currency.Type},
{
"Realized Equity",
each Number.FromText(Text.Remove(_, " "), "en-IN"),
Currency.Type
},
{"Balance", each Number.FromText(Text.Remove(_, " "), "en-US"), Currency.Type},
{"Position ID", each if _ = "-" then null else Int64.From(_), Int64.Type},
{"NWA", each Number.FromText(_, "en-US"), Currency.Type}
}
),
SplitDetails = Table.SplitColumn(
FixValsTypes, "Details", Splitter.SplitTextByEachDelimiter({"/", " "}),
{"Details", "Currency", "Split details"}
),
AddSplitValue = Table.AddColumn(
SplitDetails,
"Split value",
each
if [Split details] = null then
null
else
[
split = Text.Split([Split details], ":"),
output = Number.FromText(split{0}) / Number.FromText(split{1})
][output],
type number
),
NewType = type table Type.ForRecord(
Type.RecordFields(Type.TableRow(Value.Type(AddSplitValue)))
& [
Previous Balance = [Type = Currency.Type, Optional = false]
],
false
),
PreGroupSort = Table.Sort(
AddSplitValue, {
{"Position ID", Order.Ascending},
{"Date", Order.Ascending}
}
),
Group = Table.Group(
PreGroupSort,
{"Position ID"},
{
"Fix",
each [
group = _,
noOpen = not List.Contains( group[Type], "Open Position"),
rowOrig = group{[Type = "Open Position"]},
unitOrig = rowOrig[#"Units / Contracts"],
splitProduct = List.Product(
List.Transform(
Table.SelectRows(
group,
each [Type] = "corp action: Split"
)[Split value],
each 1 / _
)
),
unitFixed = unitOrig * splitProduct,
amountOrig = rowOrig[Amount],
amountFix = Table.ReplaceValue(
group,
each [Amount],
each
if List.Contains( {"Position closed", "corp action: Split"}, [Type])
then amountOrig
else [Amount],
Replacer.ReplaceValue,
{"Amount"}
),
rowCount = Table.RowCount(group),
addPrevBalance = Table.Buffer(
Table.FromColumns(
Table.ToColumns(amountFix) & {
{null} & List.RemoveLastN(group[Balance])
},
NewType
)
),
firstRecordFixed = Record.TransformFields(
Table.First(addPrevBalance), {{"Units / Contracts", each unitFixed ?? _}}
),
newGen = List.Generate(
() => [i = 0, rows = {firstRecordFixed}, unit = unitFixed],
each [i] < rowCount,
each let currentUnit = [unit], currentRow = addPrevBalance{[i] + 1} in
[
i = [i] + 1,
rows =
[
splitSell = Record.TransformFields(
currentRow,
{
{"Type", each "Split Sell"},
{"Units / Contracts", each currentUnit}
}
),
splitBuy = Record.TransformFields(
currentRow,
{
{"Type", each "Split Buy"},
{"Units / Contracts", each unit}
}
),
dividend = Record.TransformFields(
currentRow,
{{"Units / Contracts", each currentUnit}}
),
noChange = currentRow,
output =
if currentRow[Type] = "corp action: Split"
then {splitSell, splitBuy}
else if currentRow[Type] = "Dividend"
then {dividend}
else {noChange}
][output],
unit = [unit] * (currentRow[Split value] ?? 1)
],
each [rows]
),
comboRows = List.Combine(newGen),
output = if noOpen then group else Table.FromRecords(comboRows, NewType)
][output],
NewType
},
GroupKind.Local
),
CombineGroups = Table.Combine(Group[Fix]),
AddPricePQ = Table.AddColumn(
CombineGroups, "PricePQ",
each [Amount] / [#"Units / Contracts"],
type number
)
in
AddPricePQ
bigk
1 year agoHelper III
From the first look it seems that everything works as expected. Maybe some things will be spotted later, but as of now this case can be closed.
Thanks MarkLaf for yur help! You've saved me twice already including my other earlier request for help.
And thanks grazitti_sapna for your effort too!
PQ has limitless possibilities, unfortunately there is not enough time to learn everything. It is great to have such good community!