How to debug data flows with large data sets?

[dirk74]

I have a complex data flow with lots of steps and a very large data set of over 1 million records.

If I try to refresh the data flow, I get this error:

Error: DataFormat.Error: We couldn't convert to Number. <ccon>AV6YDBJsYJl4CHHKaSpDiHNpQ4ym0RIbuAa9Yt9ow0dQiNmMn11RfjtG8iDrsSjNJWGwjmXzfU1UvfdlwPYH1g==</ccon>. RootActivityId = 45773551-4f86-4709-b5a3-085e4a55072d.Param1 = DataFormat.Error: We couldn't convert to Number. <ccon>AV6YDBJsYJl4CHHKaSpDiHNpQ4ym0RIbuAa9Yt9ow0dQiNmMn11RfjtG8iDrsSjNJWGwjmXzfU1UvfdlwPYH1g==</ccon> Request ID: 7325f47b-5f61-f55d-a092-4a57dab751f6.

Is there any way to find the offending column/record?

Kind regards

Dirk

[lbendlin]

I have a complex data flow with lots of steps and a very large data set of over 1 million records.

Don't have lots of steps. That's not what dataflows are for.

There is no good diagnostics tool. You will have to remove steps one by one and see when the message goes away.