Forum Discussion

amaaiia's avatar
amaaiia
Skilled Sharer
1 year ago
Solved

OPTIMIZE VORDER for partitioned tables

Hi, I have a lakehouse with some tables under Tables directory. Some of the tables are partitioned by YEAR, MONTH, DAY, and other tables are not partitioned. I've applied OPTIMIZE+VORDER to all...
data engineering
lakehouse
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi amaaiia,

    Thank you for reaching out in Microsoft Community Forum.

    The reason you’re not seeing changes in partitioned tables after running OPTIMIZE and VORDER is:

    OPTIMIZE skips partitions that are already optimized, such as large or non-fragmented files.
    VORDER only triggers changes if significant data reorganization is required.

    please follow below steps to resolve this error;

    1. please Check Partition Size using below code
    DESCRIBE DETAIL delta.`/lakehouse_path/partitioned_table`

    2.Force Optimization on Specific Partitions
    OPTIMIZE delta.`/lakehouse_path/partitioned_table` WHERE YEAR = '2024'

    3. Please Verify Optimization History
    DESCRIBE HISTORY delta.`/lakehouse_path/partitioned_table`

    Please continue using Microsoft community forum.

    If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.

    Regards,
    Pavan.

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi amaaiia,

Thank you for reaching out in Microsoft Community Forum.

The reason you’re not seeing changes in partitioned tables after running OPTIMIZE and VORDER is:

OPTIMIZE skips partitions that are already optimized, such as large or non-fragmented files.
VORDER only triggers changes if significant data reorganization is required.

please follow below steps to resolve this error;

1. please Check Partition Size using below code
DESCRIBE DETAIL delta.`/lakehouse_path/partitioned_table`

2.Force Optimization on Specific Partitions
OPTIMIZE delta.`/lakehouse_path/partitioned_table` WHERE YEAR = '2024'

3. Please Verify Optimization History
DESCRIBE HISTORY delta.`/lakehouse_path/partitioned_table`

Please continue using Microsoft community forum.

If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.

Regards,
Pavan.