Need to append queries without re-pulling data from Teradata

[mtrando]

Good morning!

I am pull 6 queries from Teradata that are identical in column names, but have slightly different data. I'm pulling 6 because I'm pulling in a total of 40 million rows of data, and I can have 6 threads running at once. My download speed is very slow, about 2,000 rows per second, which is why I need 6 to run at once, which they do.

The issue is, when I want to squish the 6 into one dataset, using append, the appended query then tries to pull all the data back down from Teradata through the single thread. How can I combine query outputs without the new dataset repulling?

[VasTg]

@mtrando 

I usually disable the load for all the other 6 queries and keep "Enable Load" for the final query to improve performance.

Do remember to hide the other ones.

If this helps, mark it as a solution

Kudos are nice too.

[Tad17]

Hey @mtrando 

The functionality you're looking for is reference. References are copies of the original query that do not pull the query again. They allow you to copy the query and then make changes to it. Therefore, you can create a reference of each query and then append those to each other to save load time and avoid having to reload each query each time. 

Simply right click the query in query editor and select reference. This creates the reference table. You can do what you need to with those, create the appended query, and the ndisable load by right clicking each reference.