On Premises SQL Historical table taking long time to load into Fabric Lakehouse.

Hi devaj123,

Using Notebooks with PySpark provides the best performance and flexibility, especially for large tables. If this is a historical table that changes over time, implement watermarking and load only new or updated rows using a timestamp or identity column.

Repartition the data to spread it across multiple parallel tasks, especially helpful for performance when writing large volumes. Save the data in Delta format, which supports efficient queries, updates, and time travel versioning.

Thanks,

Prashanth Are

MS Fabric community support

If our resolved your issue, please mark it as "Accept as solution" and click "Yes" if you found it helpful.