Forum Discussion
Power Query data changes depending on rows imported
- 3 years ago
After much desparate attempts at anything and everything I some how managed to get it to function properly.
Don't know how, or why, but adjusting my SQL seem to have resolved it. My SQL was using your basic joins to expand the data, and then using a window function to limit the results for each id to the top item. This produced the expected results in SSMS (but for some reasons Power Query just would turn two columns to null unless it was under 6900 rows).
Refactoring the SQL to use a single join on a sub-query seems to work for Power Query. Here is an example of the failing query...
SELECT id, field1, field2, date1, days1, date2, days2
FROM (
SELECT id, field1, field2, apts.date1, DATEDIFF(DAY, apts.date1, CURRENT_TIMESTAMP) as days1,
date2, days2,
ROW_NUMBER() OVER (ORDER BY apts.date1) AS rn
FROM table1 tLEFT JOIN associateivetable ast ON (t.id = ast.table1id)
LEFT JOIN appointments apts ON (ast.appid = apts.appid)
) ordrd
WHERE rn = 1
After refactoring it to the following the failing columns started working..
SELECT id, field1, field2, date1, DATEDIFF(DAY, date1, CURRENT_TIMESTAMP) as days1, date2, days2
FROM table1 t
LEFT JOIN (
SELECT ast.table1id, MAX(apts.date1) AS date1
FROM associateivetable ast
LEFT JOIN appointments apts ON (ast.table1id = apts.appid)
) sq ON (t.table1id = sq.tableid)
Don't know why that worked, as I would expect results returned from SQL Server to be identical to what SSMS was getting. Perhaps some sort of AST that Power Query does to identify column type, not sure, but refactoring it to the later seemed to do the trick.
Hope that helps for anyone else who comes across this same issue.
After much desparate attempts at anything and everything I some how managed to get it to function properly.
Don't know how, or why, but adjusting my SQL seem to have resolved it. My SQL was using your basic joins to expand the data, and then using a window function to limit the results for each id to the top item. This produced the expected results in SSMS (but for some reasons Power Query just would turn two columns to null unless it was under 6900 rows).
Refactoring the SQL to use a single join on a sub-query seems to work for Power Query. Here is an example of the failing query...
SELECT id, field1, field2, date1, days1, date2, days2
FROM (
SELECT id, field1, field2, apts.date1, DATEDIFF(DAY, apts.date1, CURRENT_TIMESTAMP) as days1,
date2, days2,
ROW_NUMBER() OVER (ORDER BY apts.date1) AS rn
FROM table1 t
LEFT JOIN associateivetable ast ON (t.id = ast.table1id)
LEFT JOIN appointments apts ON (ast.appid = apts.appid)
) ordrd
WHERE rn = 1
After refactoring it to the following the failing columns started working..
SELECT id, field1, field2, date1, DATEDIFF(DAY, date1, CURRENT_TIMESTAMP) as days1, date2, days2
FROM table1 t
LEFT JOIN (
SELECT ast.table1id, MAX(apts.date1) AS date1
FROM associateivetable ast
LEFT JOIN appointments apts ON (ast.table1id = apts.appid)
) sq ON (t.table1id = sq.tableid)
Don't know why that worked, as I would expect results returned from SQL Server to be identical to what SSMS was getting. Perhaps some sort of AST that Power Query does to identify column type, not sure, but refactoring it to the later seemed to do the trick.
Hope that helps for anyone else who comes across this same issue.