Forum Discussion
Remove rows based on same AND other column
- 1 year ago
The Source line, where you see the Base64 conversion, is what you get when you paste the data into a table in PowerBI/PowerQuery. You should replace the Source line with your own source. You can read the M code documentation for the Binary.FromText function, but that Base64 argument merely determines the type of Binary encoding to be used.
And either you didn't run the code, or I misunderstood your before and after.
Source Step:
And you wrote: remove rows 1,4 and keep 2,3,5,6,7
After running code:
Did I misunderstand what you wanted? Or did you not run the code to see what it did?
If I misunderstood, what rows did you wish to keep? As the code I provided clearly keeps 2,3,5,6 and 7
- Anonymous1 year ago
Hi FabvE ,
Thanks for ronrsnfld reply.
You can try the following codelet Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndBRC4IwEAfwr3LsOVCnlj1FBX6AICrEB7WjjXSTbeLXbwwJs0HQw8F2cL/9b0VBIrIiR4WVwRPqXgqNtrG3dWGVgaqWg4F2EA3b2V4WRGFAQ5rY81mjiki5Kgh9E7lUnb1c3ThvGNxR84cArqFGY1BNCJ0j1CGxL8dPKJ5DsYMSH3RwEAoYedvCiNAhmslI50bijHS50c1WzjUDqaCX6jmNbr+fX/+zB/34kNRBGx/kz0GzRY7yBQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Unique ID" = _t, Operation = _t, #"From ID" = _t, #"Form Name" = _t, Timestamp = _t, User = _t]), AddFormID = Table.AddColumn(Source, "FormID", each if [Operation] = "CreateForm" then [From ID] else null), FillDownFormID = Table.FillDown(AddFormID,{"FormID"}), FilterCreateResponse = Table.SelectRows(FillDownFormID, each [Operation] = "CreateResponse"), AddCustom = Table.AddColumn(FilterCreateResponse, "HasCorrespondingForm", each List.Contains(FillDownFormID[FormID], [From ID])), RemoveInvalidResponses = Table.SelectRows(AddCustom, each [HasCorrespondingForm] = true), CombineTables = Table.Combine({Table.SelectRows(FillDownFormID, each [Operation] = "CreateForm"), RemoveInvalidResponses}), SortByUniqueID = Table.Sort(CombineTables,{{"Unique ID", Order.Ascending}}), RemoveHelperColumns = Table.RemoveColumns(SortByUniqueID,{"FormID", "HasCorrespondingForm"}) in RemoveHelperColumnsFinal output
Best regards,
Albert HeIf this post helps, then please consider Accept it as the solution to help the other members find it more quickly
- 1 year ago
Ok, after many tries and more fails I went with another approach which solved my problem.
I created an addinional query which contained only the created records:
let
Quelle = Table.SelectRows(Auswertung_raw, each [Operation] = "CreateForm")
in
QuelleI then used Merge queries to do a join between the initial table Auswertung_raw and the CreateForm-Table:
let
Quelle = Table.NestedJoin(Auswertung_raw, {"FormId"}, tblErstellteUmfragen, {"FormId"}, "tblErstellteUmfragen", JoinKind.LeftSemi),
#"Entfernte Duplikate" = Table.Distinct(Quelle, {"RecordId"})
in
#"Entfernte Duplikate"This gives me a table with all responses which have a corresponding CreateForm + the initial CreateForm-entries.
Nevertheless, thank you all - especially ronrsnfld- for your help and time! 👍
Hello FabvE
Your query is a bit unclear to me. It would be helpful if you could include a sample dataset with your question, as well as an example of the expected outcome. This information would greatly assist in clarifying the issue.
Thanks,
Udit
- FabvE1 year agoHelper I
Hi, I added a sample table. Hope this clarifies my question. Thx
- ronrsnfld1 year agoSuper User
I reversed your logic so we instead select rows where, for the same formId
- keep rows where operation = "CreateForm"
- if operation = "CreateResponse" then there must also be another entry with "CreateForm"
- (I did not know how to interpret your requirement surveys which were created before the first logs were delivered)
Here's one way putting your logic into a Table.SelectRows function:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("ndBRC4IwEAfwr3LsWVCnlj1FBX6AXirEB7WjjXSTbeLXbwwJs0HQw8F2cL/9b2VJYhKQk8La4Bn1IIVG2zjYurDaQN3I0UA3ipbtbS8P4yikEU3tedSoYlIFJaFvopCqt5erG+ctgztq/hDANTRoDKoZoUuEOiTx5fgJJUsocVDqg44OQgET7zqYEHpEMxvZ0kidka03utkquGYgFQxSPefR3ffzm3/2oB8fkjlo64P8OWi+ylG9AA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [uniqueId = _t, operation = _t, formId = _t, formName = _t, timestamp = _t, user = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{ {"uniqueId", Int64.Type}, {"operation", type text}, {"formId", type text}, {"formName", type text}, {"timestamp", type date}, {"user", type text}}), #"Remove Rows" = Table.SelectRows(#"Changed Type",(r)=> r[operation] = "CreateForm" or (r[operation] = "CreateResponse" and List.Contains(Table.SelectRows(#"Changed Type", each [formId]=r[formId])[operation],"CreateForm"))) in #"Remove Rows"- FabvE1 year agoHelper I
Ok, what does that base64 conversion do?
And you said:
ronrsnfld wrote:I reversed your logic so we instead select rows where, for the same formId
- keep rows where operation = "CreateForm"
- if operation = "CreateResponse" then there must also be another entry with "CreateForm"
It must be the other way round. 🙂
If there's an CreateForm there should be a CreateResponse (as a freshly created survey can have zero answers). Therefore keep all CreateForm.
But there can be formIds with CreateResponse where there's no CreateForm - formId. These should be removed.