Forum Discussion
Increased data size after new date column
- 1 year ago
Hi HR3038511
The increase in your model size is likely due to the way VertiPaq handles date columns, especially when they include both date and time components. Here are a few best practices to help reduce the size:
Disable Auto Date/Time:
Go to File > Options > Data Load and uncheck the Auto Date/Time option. This feature automatically generates hidden date tables for each date column, which can significantly inflate your model size if not required.
Split Date and Time:If your changeddate column includes both date and time components, consider splitting it into two separate columns:
One column for the date only.
Another column for the time only.
This reduces cardinality (the number of unique values) in each column, which is a key factor in model size.
Optimize Storage Format:For the date-only column, ensure the column is formatted as a Date data type, not DateTime.
If time is stored in the second column, format it as Time.
Remove Unnecessary Precision:If the time component has a precision that isn't necessary (e.g., milliseconds), round it or remove it entirely to further reduce cardinality.
Consider Aggregations:If granular date-time values aren't necessary for analysis, you could aggregate the data (e.g., by day or hour) and store only the aggregated date values.
If this post helps, then please consider Accepting it as the solution to help the other members find it more quickly.
Hello,ribisht17 ,govind_021 ,Ritaf1983 and Gabry ,thanks for your concern about this issue.
Great advice from everyone!
And I would like to share some additional solutions below.
Hi,HR3038511 .I am glad to help you.
I'll just make some brief summaries of the suggestions made by other users, and I hope you find them helpful.
If you want to reduce the impact of data columns on model capacity.
You can start from the following aspects.
1. Reduce the number of dates as much as possible, and try to use integer or other numeric types to save them, in general using integer types to save data can make occupy a large percentage of memory reduction. (provided that it does not affect the use of the date field itself)
2. Turn off the auto date/time feature in Power BI Desktop.
This is one of the official Microsoft recommended ways to reduce semantic model memory.
URL:
Data reduction techniques for Import modeling - Power BI | Microsoft Learn
I hope my suggestions give you good ideas, if you have any more questions, please clarify in a follow-up reply.
Best Regards,
Carson Jian