Forum Discussion
Missing Rows when loading Dataflow to Lakehouse
I've been encountering a peculiar issue during the loading process from Dataflow Gen2 to our Lakehouse. Although the data transformation in Gen2 appears to be successful, when I try to load the queries into our Lakehouse, only a fraction of the expected rows are being successfully loaded. Despite thorough checks on formatting and other potential issues, I haven't been able to identify any apparent problems. The loading process itself doesn't report any errors, which makes this limitation even more baffling. If anyone has faced a similar issue or has any insights into what might be causing this discrepancy, I would greatly appreciate your input and guidance.
Hi, i am yet to work on this again sorry. But i will table this issue for now after discussing with a colleague. I realized that the lakehouse was just showing a data preview, and was not supposed to show all the data. Thank you.
3 Replies
Do you happen to have a way in which we could replicate the issue? Does your query have any sort of dynamic function reference such as Date.LocalTime, Number.Random or others?
Hi frances_c
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.
Thanks.
Hi, i am yet to work on this again sorry. But i will table this issue for now after discussing with a colleague. I realized that the lakehouse was just showing a data preview, and was not supposed to show all the data. Thank you.
