Forum Discussion
Help required to separate the text
Hi Experts,
I have the following messy data and unorganized column extracted from PDF which requires separating the data in [ ] to another column.
For example,
AF-20KC - Airframe 20,000 FC Structural Check - [SN: 556] -
Result: AF-20KC - Airframe 20,000 FC Structural Check in column 1 and 556 in another column.
Also, I have these rows in which I don't want -Correction *** words. I need only upto SN: 556 and then separating into another column.
AF-192M - Airframe 192 Month Structural Check - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023
There are some rows which don't have space for the SN like below. I would like to separate SN from that row too.
EN-PR - EN-PR - Engine Full Performance Restoration
P783272]
The complete data is below:
| Description |
| AF-96M - Airframe 96 Month Structural Check - [SN: 555] |
| AF-BC - Airframe Basic Check - [SN: 555] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 555] |
| LG-OH - LG-OH - Landing Gear Leg Overhaul |
| AF-192M - Airframe 192 Month Structural Check - [SN: 555] |
| AP-OH - AP-OH - APU Overhaul |
| EN-PR - EN-PR - Engine Full Performance Restoration P783243] |
| EN-LP - EN-LP - Engine LLP Replacement |
| EN-PR - EN-PR - Engine Full Performance Restoration P783245] |
| LG-OH - LG-OH - Landing Gear Leg Overhaul - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AF-192M - Airframe 192 Month Structural Check - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AF-BC - Airframe Basic Check - [SN: 556] - Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AF-96M - Airframe 96 Month Structural Check - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 556] - |
| Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AP-OH - AP-OH - APU Overhaul - [SN: PWC-ZD0103] - Correction from period 01-Dec-2023 to 31-Dec-2023 |
| - [SN: EN-LP - EN-LP - Engine LLP Replacement P783246]Correction from period 01-Dec-2023 to 31-Dec-2023 |
| - [SN: EN-PR - EN-PR - Engine Full Performance Restoration P783246] - Correction from period 01-Dec-2023 to 31-Dec-2023 |
| - [SN: P783255] EN-LP - EN-LP - Engine LLP Replacement Correction from period 01-Dec-2023 to 31-Dec-2023 |
| - [SN: EN-PR - EN-PR - Engine Full Performance Restoration P783255] - Correction from period 01-Dec-2023 to 31-Dec-2023 |
| AF-96M - Airframe 96 Month Structural Check - [SN: 557] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 557] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 557] |
| AF-BC - Airframe Basic Check - [SN: 557] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783258] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783259] |
| AF-BC - Airframe Basic Check - [SN: 558] |
| AF-96M - Airframe 96 Month Structural Check - [SN: 558] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 558] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 558] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783257] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783260] |
| AF-96M - Airframe 96 Month Structural Check - [SN: 561] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 561] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 561] |
| AF-BC - Airframe Basic Check - [SN: 561] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783274] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783277] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 559] |
| AF-96M - Airframe 96 Month Structural Check - [SN: 559] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 556] |
| AF-BC - Airframe Basic Check - [SN: 556] |
| AF-96M - Airframe 96 Month Structural Check - [SN: 556] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 556] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783246] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783255] |
| AF-96M - Airframe 96 Month Structural Check - [SN: 560] |
| AF-192M - Airframe 192 Month Structural Check - [SN: 560] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 560] |
| AF-BC - Airframe Basic Check - [SN: 560] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783247] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783271] |
| AF-20KC - Airframe 20,000 FC Structural Check - [SN: 559] |
| AF-BC - Airframe Basic Check - [SN: 559] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783272] |
| EN-PR - EN-PR - Engine Full Performance Restoration P783273] @ |
10 Replies
- dufoq3Community Champion
Hi vineshparekh,
check whether this is what you need.
Result (top 20 rows)
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("xVdbb4IwFP4rJz6PpIIU2NMUp0vmhWiWJTN9aLAqGYKpuN8/hCm4S7ScJnuhqfR837l9p7hYtLoDw6NjMKAbyZXkWwEehXGaZBuYZ/IQZgfJY/A3InzPDy3mk3uwbZu12F1h2/Prpj2+j8K/D5vk+eK4Se4IITDwr1ONhsb0Kf/1vPJkGSVrGAouYSTWMP0QcsMP8Ymr7ZkXUeX7W8PixfNxYgSz/N15TdZRImBwiGMIhFylcsuTUMBM7LNU8ixKk8IucFzL7FjsBDIKSpByLUFG+WYmdjEPxVYkmQY+1TRVQVMGhp9KKcIjJKxkuoWdkFG6BNI2+iLMy2ZakKVgVVtMllGEN/TbER++ERTmN5OoCwIVUxNZHAkL+4asQdkf1fryozWCV99465M2sS4TWgAocH3BKUihbGnKmsVW8SH09FsTqfEXOPlIUQn8/wI+Oto04EaKcRhmhjioO8VRub4chh/PtqsDxFNx22XNa+OiauOiaqMlUzpqRknzDNI2JoM16wYZrFlfb5PTYVSmnI4OENxA8BDt7qGYqYoqKcJNitIV1VCjjg6Q2v8BdV0RlK4ISldERVdER7p1jDEHNUzU7hxPh7+mDpD8k/Whxdgn", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Description = _t]), AddedIndex = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type), Ad_GroupHelper = Table.AddColumn(AddedIndex, "GroupHelper", each if (try Text.Range([Description], 2, 1) otherwise null) = "-" then [Index] else null, type text), FilledDown = Table.FillDown(Ad_GroupHelper,{"GroupHelper"}), GroupedRows = Table.Group(FilledDown, {"GroupHelper"}, {{"Description", each Text.Combine([Description], " "), type text}}), ExtractedTextBeforeDelimiter = Table.TransformColumns(GroupedRows, {{"Description", each [ a = Text.PositionOf(_, "]", Occurrence.First) +1, b = Text.Range(_, 0, if a > 0 then a else null ) ][b], type text}}), TrimmedText = Table.TransformColumns(ExtractedTextBeforeDelimiter,{{"Description", Text.Trim, type text}}), Ad_SN = Table.AddColumn(TrimmedText, "SN", each [ a = Text.PositionOf([Description], "[SN") +1, b = Text.PositionOf([Description], " ", Occurrence.Last), c = Text.Range([Description], if a > 0 then a else if Text.Contains([Description], "]") then b else Text.Length([Description])) ][c], type text), CleanDescription = Table.ReplaceValue(Ad_SN, each [SN], null, (x,y,z)=> if y = "" then x else Text.TrimEnd(Text.Replace(x, y, ""), {"[", " ", "-"}), {"Description"}), CleanSN = Table.ReplaceValue(CleanDescription, null, null, (x,y,z)=> if x = "" then null else Text.Replace(Text.Trim(x, {" ", "]"}), "SN: ", ""), {"SN"}), RemovedGroupHelper = Table.RemoveColumns(CleanSN,{"GroupHelper"}) in RemovedGroupHelper- vineshparekhHelper I
Hi dufoq3
Thank you for the response!
I am getting this error when I use the M code you provided. This is coming from Ad_GroupHelper step.Expression.Error: The 'offset' argument is out of range.
- dufoq3Community Champion
vineshparekh, I've edited the code above. Try it now.
- vineshparekhHelper I
dufoq3 Thanks! It works now. Just last thing!
"Restoration" word for the Engine Full Performance is going into the second column as well. Whereas, I would like it to remain in the first column.
EN-PR - EN-PR - Engine Full Performance Restoration
P783243- dufoq3Community Champion
There is nothing like that in sample data. You have to provide new sample data with this issue.
- vineshparekhHelper I
Sample data does have those rows. For example, if you search this text in Bold in the dataset, you will find the related row. I am also providing the sample data again.
EN-PR - EN-PR - Engine Full Performance Restoration
P783277]One additional question.
The sample data I had provided had only one column but in my actual data set, I have multiple columns which I want to populate in the final step. Is it the group by where the column names have to be added to show them as well? Please review my sample data.
DescriptionUtilisationColumn4RateAmount AF-96M - Airframe 96 Month Structural Check - [SN: 555] 1.00 MTH 702.00 702.00 AF-BC - Airframe Basic Check - [SN: 555] 241.20 FH 37.00 8,924.40 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 555] 95.00 FC 4.40 418.00 LG-OH - LG-OH - Landing Gear Leg Overhaul 1.00 MTH 2,652.00 2,652.00 AF-192M - Airframe 192 Month Structural Check - [SN: 555] 1.00 MTH 610.00 610.00 AP-OH - AP-OH - APU Overhaul 95.10 APUH 25.00 2,377.50 EN-PR - EN-PR - Engine Full Performance Restoration
P783243]241.20 FH 135.00 32,562.00 EN-LP - EN-LP - Engine LLP Replacement 95.00 FC 162.08 15,397.60 EN-PR - EN-PR - Engine Full Performance Restoration
P783245]241.20 FH 135.00 32,562.00 LG-OH - LG-OH - Landing Gear Leg Overhaul - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 1.00 2,652.00 2,652.00 AF-192M - Airframe 192 Month Structural Check - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 1.00 610.00 610.00 AF-BC - Airframe Basic Check - [SN: 556] - Correction from
period 01-Dec-2023 to 31-Dec-2023260.23 37.00 9,628.51 AF-96M - Airframe 96 Month Structural Check - [SN: 556] -Correction from period 01-Dec-2023 to 31-Dec-2023 1.00 702.00 702.00 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 556] - 99.00 4.40 435.60 Correction from period 01-Dec-2023 to 31-Dec-2023 AP-OH - AP-OH - APU Overhaul - [SN: PWC-ZD0103] - Correction
from period 01-Dec-2023 to 31-Dec-202390.30 25.00 2,257.50 - [SN: EN-LP - EN-LP - Engine LLP Replacement
P783246]Correction from period 01-Dec-2023 to 31-Dec-202399.00 162.08 16,045.92 - [SN: EN-PR - EN-PR - Engine Full Performance Restoration
P783246] - Correction from period 01-Dec-2023 to 31-Dec-2023260.23 135.00 35,131.05 - [SN: P783255] EN-LP - EN-LP - Engine LLP Replacement
Correction from period 01-Dec-2023 to 31-Dec-202399.00 162.08 16,045.92 - [SN: EN-PR - EN-PR - Engine Full Performance Restoration
P783255] - Correction from period 01-Dec-2023 to 31-Dec-2023260.23 135.00 35,131.05 AF-96M - Airframe 96 Month Structural Check - [SN: 557] 0.90 MTH 702.00 631.80 AF-192M - Airframe 192 Month Structural Check - [SN: 557] 0.90 MTH 610.00 549.00 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 557] 2.00 FC 4.40 8.80 AF-BC - Airframe Basic Check - [SN: 557] 11.08 FH 37.00 409.96 EN-PR - EN-PR - Engine Full Performance Restoration
P783258]6.90 APUH 25.00 172.50 EN-PR - EN-PR - Engine Full Performance Restoration
P783259]11.08 FH 135.00 1,495.80 AF-BC - Airframe Basic Check - [SN: 558] 2.00 FC - - AF-96M - Airframe 96 Month Structural Check - [SN: 558] 11.08 FH 135.00 1,495.80 AF-192M - Airframe 192 Month Structural Check - [SN: 558] 2.00 FC - - AF-20KC - Airframe 20,000 FC Structural Check - [SN: 558] 0.90 MTH 2,652.00 2,386.80 EN-PR - EN-PR - Engine Full Performance Restoration
P783257]110.67 FH 37.00 4,094.79 EN-PR - EN-PR - Engine Full Performance Restoration
P783260]0.67 MTH 702.00 470.34 AF-96M - Airframe 96 Month Structural Check - [SN: 561] 0.67 MTH 610.00 408.70 AF-192M - Airframe 192 Month Structural Check - [SN: 561] 40.00 FC 4.40 176.00 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 561] 49.00 APUH 25.00 1,225.00 AF-BC - Airframe Basic Check - [SN: 561] 110.67 FH 135.00 14,940.45 EN-PR - EN-PR - Engine Full Performance Restoration
P783274]40.00 FC - - EN-PR - EN-PR - Engine Full Performance Restoration
P783277]110.67 FH 135.00 14,940.45 AF-192M - Airframe 192 Month Structural Check - [SN: 559] 40.00 FC - - AF-96M - Airframe 96 Month Structural Check - [SN: 559] 0.67 MTH 2,652.00 1,776.84 AF-192M - Airframe 192 Month Structural Check - [SN: 556] 0.43 MTH 702.00 301.86 AF-BC - Airframe Basic Check - [SN: 556] 0.43 MTH 610.00 262.30 AF-96M - Airframe 96 Month Structural Check - [SN: 556] 17.00 FC 4.40 74.80 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 556] 55.22 FH 37.00 2,043.14 EN-PR - EN-PR - Engine Full Performance Restoration
P783246]29.20 APUH 25.00 730.00 EN-PR - EN-PR - Engine Full Performance Restoration
P783255]55.22 FH 135.00 7,454.70 AF-96M - Airframe 96 Month Structural Check - [SN: 560] 17.00 FC - - AF-192M - Airframe 192 Month Structural Check - [SN: 560] 55.22 FH 135.00 7,454.70 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 560] 17.00 FC - - AF-BC - Airframe Basic Check - [SN: 560] 0.43 MTH 2,652.00 1,140.36 EN-PR - EN-PR - Engine Full Performance Restoration
P783247]0.10 610.00 61.00 EN-PR - EN-PR - Engine Full Performance Restoration
P783271]0.10 702.00 70.20 AF-20KC - Airframe 20,000 FC Structural Check - [SN: 559] 0.10 2,652.00 265.20 AF-BC - Airframe Basic Check - [SN: 559] 1.00 MTH 723.06 723.06 EN-PR - EN-PR - Engine Full Performance Restoration
P783272]224.48 FH 38.11 8,554.93 EN-PR - EN-PR - Engine Full Performance Restoration
P783273]85.00 FC 4.53 385.05