Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago

applying filters in the dataflow

Hello everyone,
I am using Dataflow to extract data from a table with more than 550 million records.
I'm getting an error message regarding timed out.
I thought about separating the old data that will no longer be updated, filtering through the date field to load them the first time, and then doing an incremental process to load the new data.
I did a test with only one month, about 90 million records, and the same error continues to occur.
Are the filters applied in the dataflow not applied when updating the data?

2 Replies