Forum Discussion
Helper IIIInquiry about file size difference between Widow and Fabric
The file size shown in Widow and the file size of the data upload are different.
If I look at the file size of test2.csv on Windows it's 1.81GB, but if I try to upload it from Fabric it's 2.28GB.
Why is that?
Hi leesukjun
The difference in size (1.81 GB on Windows vs. 2.28 GB in Fabric) is primarily due to how Microsoft Fabric processes and stores files after upload. It involves transformations such as adding metadata, changing storage formats, or applying encoding adjustments that make the uploaded version larger than its original representation on Windows.
Hi leesukjun
nilendraFabric provides a very meaningful explanation.
Windows usually uses NTFS, which has its own way of storing and compressing files. Fabric mainly relies on OneLake, a unified data lake storage system. OneLake uses the open Delta Lake format to store data. This format allows efficient data storage and processing, and supports a variety of data analysis and machine learning workloads.Therefore, these are two different ways, which may cause changes in file size when uploading.When files are uploaded to Fabric, additional encoding and metadata may be added to ensure compatibility and integrity within the Fabric environment. This may include information about the file structure, permissions, and other properties.
Fabric may transform the data to optimize its processing needs. This transformation may include changes in the data format, structure, or encoding, which can increase the file size.
This information is not visible in the Windows file size.
Knowing these details can help you explain the difference to your clients and that the increased file size is a normal part of the upload process to Fabric.
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
4 Replies
- nilendraFabric
Super User
Hi leesukjun
The difference in size (1.81 GB on Windows vs. 2.28 GB in Fabric) is primarily due to how Microsoft Fabric processes and stores files after upload. It involves transformations such as adding metadata, changing storage formats, or applying encoding adjustments that make the uploaded version larger than its original representation on Windows.
Hi leesukjun
nilendraFabric provides a very meaningful explanation.
Windows usually uses NTFS, which has its own way of storing and compressing files. Fabric mainly relies on OneLake, a unified data lake storage system. OneLake uses the open Delta Lake format to store data. This format allows efficient data storage and processing, and supports a variety of data analysis and machine learning workloads.Therefore, these are two different ways, which may cause changes in file size when uploading.When files are uploaded to Fabric, additional encoding and metadata may be added to ensure compatibility and integrity within the Fabric environment. This may include information about the file structure, permissions, and other properties.
Fabric may transform the data to optimize its processing needs. This transformation may include changes in the data format, structure, or encoding, which can increase the file size.
This information is not visible in the Windows file size.
Knowing these details can help you explain the difference to your clients and that the increased file size is a normal part of the upload process to Fabric.
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- leesukjun
Thank you for your reply.
Do you have any more detailed information?
If I don't know the information about how to save the file, I can't convince my client.
Should I contact MS, not the community, for this?
Do you have any detailed information about this?
Thank you.- nilendraFabric
