Forum Discussion
adding rows in power query (and performance issue)
- 8 years ago
Hi Sam12,
Please try this code in the advanced query editor after exporting your data and arrange the corresponding fields:
- First I have field Date and Amount ( from your raw data)
1) Make sure your Date is type Date
2) Create an index
3) Find the difference of day(s) between consecutive date
4) Create a list of date
5) Expand the list created and removed the unnecessary fields ( index, customs)
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content], #"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Amount", Int64.Type}}), #"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 1, 1), #"Added Custom" = Table.AddColumn(#"Added Index", "Custom", each try #"Added Index"[Date]{[Index]}-[Date] otherwise 1), #"Changed Type1" = Table.TransformColumnTypes(#"Added Custom",{{"Custom", Int64.Type}}), #"Added Custom1" = Table.AddColumn(#"Changed Type1", "Custom.1", each List.Dates(Date.From([Date]),[Custom],#duration(1,0,0,0))), #"Expanded Custom.1" = Table.ExpandListColumn(#"Added Custom1", "Custom.1"), #"Reordered Columns" = Table.ReorderColumns(#"Expanded Custom.1",{"Custom.1", "Amount", "Date", "Index", "Custom"}), #"Removed Other Columns" = Table.SelectColumns(#"Reordered Columns",{"Custom.1", "Amount"}), #"Renamed Columns" = Table.RenameColumns(#"Removed Other Columns",{{"Custom.1", "Date"}}) in #"Renamed Columns"
Hi Sam12 ,
I have never seen this message>
But what about checking the field ( removing null or replacing by a constant) and then proceed....
Please let me know if it does not help, and we can figure out something different.
Ninter
Hi, i just posted the message.
I tried to check this (null) but scolling through 65k rows is not that fast in PQ. Is there a way to extract it from PQ? Like copy-paste into Excel with all rows and not only 999 rows? Let me know.
P.
- Interkoubess8 years agoSolution Sage
Hi Sam12,
Yeah you can extract all your data from PQ simply by close-and-load ( In the top left corner ).
Please let me know if I missed something.
But what about replacing the null value in PQ, it did not work?
Thank you.
Ninter
- Sam128 years agoRegular Visitor
Hi, I did a replace, but it did not have an effect, still errors.
What about splitting the data set in smaller lists of approx 10000 rows (original file is 65000 row: 500products*130rows), and apply the proposed script. We can make sub selections using another table that states which products to include (innerjoin?)
Can we append the queries afterwards back to the total list again of 65000 rows? Does this reduce the load on the system?
thanks for you ideas.
P.