Forum Discussion
Building a File based data mart strategy?
Are you bringing the data into directly from SharePoint to a fabric data warehouse into individual tables like I described?We are not there yet - we are using SharePoint as our Data Lake for now. But with Data Factory you can bring the data over if you need to.
we cannot easily manage this in SharePoint to PowerBI any moreCan you please elaborate? With a sufficient quota a SharePoint Document Library seems to work well for us, especially when you can avoid folder contents enumeration across the whole site.
A Fabric Data Lake would be an obvious choice but it is based on Parquet (Delta Lake) files, and the general guidance is to avoid the "many small files" scenario that you have. They prefer large files in the gigabyte range.
If you can run a Fabric trial you should test it out and see what the performance is, including the transforms from CSV to Parquet (which is built in, I think).
Are you bringing the data into directly from SharePoint to a fabric data warehouse into individual tables like I described? If that was possible and then we could do transformations in the data warehouse that would work
Are you bringing the data into directly from SharePoint to a fabric data warehouse into individual tables like I described?We are not there yet - we are using SharePoint as our Data Lake for now. But with Data Factory you can bring the data over if you need to.
