Forum Discussion
Resolver ICombined dataflow - most rows missing
I'd start by checking the row count after each step in the combined dataflow, especially the filter on part number. Even a simple filter can remove more rows than expected if some values are null or formatted differently.
Also, are you appending the 9 tables or merging them? If you're merging, the join type could explain why so many rows are missing.
The refresh time being under 2 hours doesn't necessarily mean all the rows were loaded. Does the refresh history show any warnings?
Resolver IThanks for the reply, JD.
Doing a row count seem like a sensible thing to try, thanks.
It is an append-query.
Refresh history show no warnings.
I have checked the data type of the part number-column which is the same in all source tables.
Any additional inputs from anyone are welcome.
- ShivekMaharaj
Community Champion
Hi 1up,
Since this is an Append query, I would first prove whether the row loss happens before or after the append itself.
Microsoft's Append queries guidance confirms that append simply adds the rows from the selected tables together based on column names. It does not behave like a join, so it should not silently drop millions of rows just because the source schemas differ.
I would therefore add a temporary row count to each of the 9 source queries inside the combined dataflow before the append. If one source already shows only tens of thousands of rows there, then the append is not the cause.
At that point I would compare that source query with the original dataflow and check whether the combined dataflow is reading a linked/dependent table, applying an incremental refresh policy, or evaluating different upstream steps.
Microsoft's dataflow refresh troubleshooting guidance specifically recommends checking the refresh history CSV for Rows processed and Bytes processed when a dataflow refresh succeeds but produces much less data than expected.
That would give you a useful comparison:
- original source dataflow row count
- row count when referenced inside the combined dataflow
- row count immediately after Table.Combine
- row count after each filtering step
If the count only drops after Table.Combine, that would be unexpected and worth isolating further. If it is already low before the append, I would focus on the source/dependency refresh path rather than the append logic itself.I would also check whether incremental refresh is enabled anywhere in the chain, because Microsoft notes that dependent or linked dataflows can behave differently depending on the upstream refresh state.
So my first diagnostic would be to stop looking at the final 40 million rows and identify the first exact step where one source changes from millions to thousands.
AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
