Forum Discussion

christianunr's avatar
christianunr
Helper I
1 year ago
Solved

Copy Activity is very slow

I have a data pipeline that has a Copy Activity action. The data that is being pulled from our on-prem SQL Server is tiny. The biggest table is about 1,800. However, despite the small size, Copy Acti...
data pipeline
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi christianunr,

    Thank you for reaching out in Microsoft Community Forum.

    Yes, OPTIMIZE is a SQL command designed for Delta tables in Fabric Lakehouse. It helps improve query and write performance by compacting small files into larger ones, reducing metadata overhead
    .

    You can run it in a SQL notebook or a SQL query execution environment within Fabric:
    OPTIMIZE delta.`/lakehouse_path/table_name`

    Please continue using Microsoft community forum.

    If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.

    Regards,
    Pavan.

christianunr's avatar
christianunr
Helper I
1 year ago

Hi Anonymous - for option #1, is the OPTIMIZE a SQL command?

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi christianunr,

Thank you for reaching out in Microsoft Community Forum.

Yes, OPTIMIZE is a SQL command designed for Delta tables in Fabric Lakehouse. It helps improve query and write performance by compacting small files into larger ones, reducing metadata overhead
.

You can run it in a SQL notebook or a SQL query execution environment within Fabric:
OPTIMIZE delta.`/lakehouse_path/table_name`

Please continue using Microsoft community forum.

If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.

Regards,
Pavan.