Forum Discussion
Need Advise On SharePoint to Datamart or Dataflow
So i need advise where like there is a situation my source data is from SharePoint site and previously by building report i connected with shraepoint only the data gets added monthly now when in a particular month the data became so large that it couldnt fit in a single excel sheet so they made or splitted it and while adding new data the schema also changed inconsitent columns and header names and son on now the data some of the data is missing like some month and now they are saying that the data will become exponentialy larger so the previous etl in power query which had done is also slowing and refresh is also taking very much time so i was thinking to you know get the data from sharepoint using dataflow and dump it in a datamart for better performance and scalability ....... will it be a good solution anything would help or any alternate methods as a fresher just into this itindustry and this tool i dont have that much idea by reading the microsoft forum and learn material i came to this idea ...... any better methods or is this viable solution would help
Thanks in Advance
- Anonymous1 year ago
Hi Akash_Varuna,
Thank you for reaching out to Microsoft Fabric Community Forum.
Your idea to use Dataflows to pull in and preprocess SharePoint data, then load it into a Datamart is a good approach. However, i would like to suggest you to use Fabric Lakehouses to store SharePoint data. It efficiently handle large volumes of data using Delta Lake storage.
Did I answer your question? Mark my post as a solution, this will help others!
If my response(s) assisted you in any way, don't forget to give "Kudos"Regards,
Vinay pabbu
2 Replies
- AnonymousNot applicable
Hi Akash_Varuna,
Thank you for reaching out to Microsoft Fabric Community Forum.
Your idea to use Dataflows to pull in and preprocess SharePoint data, then load it into a Datamart is a good approach. However, i would like to suggest you to use Fabric Lakehouses to store SharePoint data. It efficiently handle large volumes of data using Delta Lake storage.
Did I answer your question? Mark my post as a solution, this will help others!
If my response(s) assisted you in any way, don't forget to give "Kudos"Regards,
Vinay pabbu
- Akash_Varuna
Super User
Thanks for the confirmation and suggestion actually in my current organisation they havent yet adopted MS Fabric yet but that suggestion widened my view thanks again