Strategies for Batch Processing in Fabric Data Warehouse without Temporary Tables

[smoqt]

I'm currently working on implementing a merging strategy within a Fabric Data Warehouse environment, where I need to merge data from a stage table to a target table.

In my Azure SQL workflows, I typically use temporary tables for batch processing.  However, my understanding is that temporary tables are not supported in Fabric Data Warehouse.

I could use CTEs, but their ephemeral nature poses some challenges.  I could also persist tables and drop them once they are no longer needed.  What else?

[AndyDDC · Accepted solution]

Hi @smoqt there are not that many options in the Fabric Warehouse service itself in terms of holding data for processing while using SQL.  I would use persisted tables and drop them after they are no longer used.

[smoqt]

Thanks for the info.  I wanted to make sure I wasn't missing anything.