Forum Discussion
Remove rows based on same AND other column
- 1 year ago
The Source line, where you see the Base64 conversion, is what you get when you paste the data into a table in PowerBI/PowerQuery. You should replace the Source line with your own source. You can read the M code documentation for the Binary.FromText function, but that Base64 argument merely determines the type of Binary encoding to be used.
And either you didn't run the code, or I misunderstood your before and after.
Source Step:
And you wrote: remove rows 1,4 and keep 2,3,5,6,7
After running code:
Did I misunderstand what you wanted? Or did you not run the code to see what it did?
If I misunderstood, what rows did you wish to keep? As the code I provided clearly keeps 2,3,5,6 and 7
- Anonymous1 year ago
Hi FabvE ,
Thanks for ronrsnfld reply.
You can try the following codelet Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndBRC4IwEAfwr3LsOVCnlj1FBX6AICrEB7WjjXSTbeLXbwwJs0HQw8F2cL/9b0VBIrIiR4WVwRPqXgqNtrG3dWGVgaqWg4F2EA3b2V4WRGFAQ5rY81mjiki5Kgh9E7lUnb1c3ThvGNxR84cArqFGY1BNCJ0j1CGxL8dPKJ5DsYMSH3RwEAoYedvCiNAhmslI50bijHS50c1WzjUDqaCX6jmNbr+fX/+zB/34kNRBGx/kz0GzRY7yBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Unique ID" = _t, Operation = _t, #"From ID" = _t, #"Form Name" = _t, Timestamp = _t, User = _t]), AddFormID = Table.AddColumn(Source, "FormID", each if [Operation] = "CreateForm" then [From ID] else null), FillDownFormID = Table.FillDown(AddFormID,{"FormID"}), FilterCreateResponse = Table.SelectRows(FillDownFormID, each [Operation] = "CreateResponse"), AddCustom = Table.AddColumn(FilterCreateResponse, "HasCorrespondingForm", each List.Contains(FillDownFormID[FormID], [From ID])), RemoveInvalidResponses = Table.SelectRows(AddCustom, each [HasCorrespondingForm] = true), CombineTables = Table.Combine({Table.SelectRows(FillDownFormID, each [Operation] = "CreateForm"), RemoveInvalidResponses}), SortByUniqueID = Table.Sort(CombineTables,{{"Unique ID", Order.Ascending}}), RemoveHelperColumns = Table.RemoveColumns(SortByUniqueID,{"FormID", "HasCorrespondingForm"}) in RemoveHelperColumnsFinal output
Best regards,
Albert HeIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly
- 1 year ago
Ok, after many tries and more fails I went with another approach which solved my problem.
I created an addinional query which contained only the created records:
let
Quelle = Table.SelectRows(Auswertung_raw, each [Operation] = "CreateForm")
in
QuelleI then used Merge queries to do a join between the initial table Auswertung_raw and the CreateForm-Table:
let
Quelle = Table.NestedJoin(Auswertung_raw, {"FormId"}, tblErstellteUmfragen, {"FormId"}, "tblErstellteUmfragen", JoinKind.LeftSemi),
#"Entfernte Duplikate" = Table.Distinct(Quelle, {"RecordId"})
in
#"Entfernte Duplikate"This gives me a table with all responses which have a corresponding CreateForm + the initial CreateForm-entries.
Nevertheless, thank you all - especially ronrsnfld- for your help and time! 👍
What you need to do is run your own query until you have a table which looks like your data table that you posted in your original question. Then, in the query I provided, you should be able to reference your own query in the Source line.
eg: Source = #"Your Query Name"
Well I combined your code with mine and it's like this now:
let
Quelle = Table.Combine({#"2023-10_12", #"2024-01_05", #"2024-06"}),
#"Remove Duplicates" = Table.Distinct(Quelle, {"RecordId"}),
#"Get JSON" = Table.AddColumn(#"Remove Duplicates", "JSON Block", each Json.Document([AuditData])),
#"Extend JSON Block" = Table.ExpandRecordColumn(#"Get JSON", "JSON Block", {"FormId", "FormName", "SourceApp", "FormsUserType", "OrganizationId"}, {"JSON Block.FormId", "JSON Block.FormName", "JSON Block.SourceApp", "JSON Block.FormsUserType", "JSON Block.OrganizationId"}),
#"Remove Columns" = Table.RemoveColumns(#"Extend JSON Block",{"AuditData"}),
#"Column isAnonymous" = Table.AddColumn(#"Remove Columns", "isAnonymous", each if Text.StartsWith([UserId], "urn:forms:anonymous") then true else false, type logical),
#"Sort Rows" = Table.Sort(#"Column isAnonymous",{{"CreationDate", Order.Descending}}),
#"SourceApp Mapping" = fBulkReplace(#"Sort Rows", tblSourceAppMapping, {"JSON Block.SourceApp"}),
#"FormsUserType Mapping" = fBulkReplace(#"SourceApp Mapping", tblFormsUserType, {"JSON Block.FormsUserType"}),
#"Remove Rows" = Table.SelectRows(#"FormsUserType Mapping",(r)=>
r[Operation] = "CreateForm"
or (r[Operation] = "CreateResponse"
and List.Contains(Table.SelectRows(#"FormsUserType Mapping", each [JSON Block.FormId]=r[JSON Block.FormId])[Operation],"CreateForm")))
in
#"Remove Rows"
It's running over 30mins now and doesn't seem to end...
When I select the step before (FormsUserType Mapping) it's quiet fast but the last step doesn't come to an end.
- ronrsnfld1 year agoSuper User
But that step "FormsUserType Mapping" is not part of the code I supplied. It's something you've developed; it refers to a function you've developed, so I don't know how to help.
As I suggested above, it would be better (at least for troubleshooting) to split the queries and have mine just refer to yours. Then, if mine is still running slowly, you could supply a more realistic example so we can see where the hangup is.