Forum Discussion
AWS S3 shortcut issue
Hello Everyone,
I have my AWS S3 bucket with partitioned data by year, month and day in csv format
- some files are pipe delimited and some some files are comma delimited.
- there are data starting with ^ and ending with ^
- When creating shortcut on tables it is not applied on all files when i click on root folder, i have drill through down the leaf and select all the files individually
Because of the above two points im not able to create shortcut on tables directly, under files i can create shortcut but while converting that to tables again same error. For point 1 i can correct from source side, but for point 2 is there any solution?
Is there recommendation for the above issue?
Regards,
Srisakthi
Hi Srisakthi,
When you create a shortcut in Fabric, all the files in that folder need to follow the same structure for example, same delimiter and schema. Since some of your files are pipe-delimited and some are comma-delimited, Fabric is treating them as different formats, which is why the shortcut isn’t applying at the root level.For this scenario, the recommendation is to make sure the files are standardized before creating the shortcut same delimiter and structure. Once the files are uniform, you should be able to create the shortcut at the folder level without drilling down into each file.
Kindly refer to the below documentation links for better understanding:
https://learn.microsoft.com/en-us/fabric/onelake/shortcuts-file-transformations/transformations
https://learn.microsoft.com/en-us/fabric/onelake/create-onelake-shortcut
https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcutsHope this helps if you have any queries we are happy to assist you further.
Best Regards,
Harshitha.
4 Replies
Hi Srisakthi,
Thank you for reaching out to the Microsoft fabric community forum.
Just to understand better, are you facing this issue while connecting to your AWS S3 data through Power BI or when working with Microsoft Fabric? This will help us guide you with the right steps.
Best Regards,
Harshitha.Hi Srisakthi,
When you create a shortcut in Fabric, all the files in that folder need to follow the same structure for example, same delimiter and schema. Since some of your files are pipe-delimited and some are comma-delimited, Fabric is treating them as different formats, which is why the shortcut isn’t applying at the root level.For this scenario, the recommendation is to make sure the files are standardized before creating the shortcut same delimiter and structure. Once the files are uniform, you should be able to create the shortcut at the folder level without drilling down into each file.
Kindly refer to the below documentation links for better understanding:
https://learn.microsoft.com/en-us/fabric/onelake/shortcuts-file-transformations/transformations
https://learn.microsoft.com/en-us/fabric/onelake/create-onelake-shortcut
https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcutsHope this helps if you have any queries we are happy to assist you further.
Best Regards,
Harshitha.
