Forum Discussion
OPTIMIZE VORDER for partitioned tables
Hi,
I have a lakehouse with some tables under Tables directory.
Some of the tables are partitioned by YEAR, MONTH, DAY, and other tables are not partitioned.
I've applied OPTIMIZE+VORDER to all the tables and I only see changes in the NOT partitioned tables. I see a new parquet file for theese tables, but partitioned tables have not changes in _delta_log directory nor new parquet file.
Why??
Hi amaaiia,
Thank you for reaching out in Microsoft Community Forum.
The reason you’re not seeing changes in partitioned tables after running OPTIMIZE and VORDER is:
OPTIMIZE skips partitions that are already optimized, such as large or non-fragmented files.
VORDER only triggers changes if significant data reorganization is required.
please follow below steps to resolve this error;
1. please Check Partition Size using below code
DESCRIBE DETAIL delta.`/lakehouse_path/partitioned_table`2.Force Optimization on Specific Partitions
OPTIMIZE delta.`/lakehouse_path/partitioned_table` WHERE YEAR = '2024'3. Please Verify Optimization History
DESCRIBE HISTORY delta.`/lakehouse_path/partitioned_table`Please continue using Microsoft community forum.
If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.
Regards,
Pavan.
1 Reply
Hi amaaiia,
Thank you for reaching out in Microsoft Community Forum.
The reason you’re not seeing changes in partitioned tables after running OPTIMIZE and VORDER is:
OPTIMIZE skips partitions that are already optimized, such as large or non-fragmented files.
VORDER only triggers changes if significant data reorganization is required.
please follow below steps to resolve this error;
1. please Check Partition Size using below code
DESCRIBE DETAIL delta.`/lakehouse_path/partitioned_table`2.Force Optimization on Specific Partitions
OPTIMIZE delta.`/lakehouse_path/partitioned_table` WHERE YEAR = '2024'3. Please Verify Optimization History
DESCRIBE HISTORY delta.`/lakehouse_path/partitioned_table`Please continue using Microsoft community forum.
If you found this post helpful, please consider marking it as "Accept as Solution" and give it a 'Kudos'. if it was helpful. help other members find it more easily.
Regards,
Pavan.