Forum Discussion
How to combine SharePoint List from multiple SharePoint sites
- Anonymous2 years ago
I cannot share the pbix sample but I can provide the explanation of the code:
The code is designed to fetch data from multiple SharePoint lists based on the URLs and list names provided in an Excel file.
The first part of the code reads an Excel file. This Excel file contains a list of SharePoint site URLs and corresponding list names. The Excel.Workbook function is used to load the Excel file, and the Table.RenameColumns function is used to rename the columns to "Site URL" and "List Name".
Next, a custom function named FetchList is defined. This function takes a SharePoint site URL as an argument and returns a table of all lists in the SharePoint site. The SharePoint.Tables function is used to fetch the lists from the SharePoint site.
Finally, a new column named "Custom" is added to the table. For each row in the table, the FetchList function is called with the site URL from that row. The result is a table with an additional column that contains the SharePoint lists for each site URL.
This code is a common pattern in Power Query M language for fetching and combining data from multiple sources based on a list of parameters.
Hello Rubayart,
Thanks for quick response. I already followed these steps. As i mentioned earlier that I am maintain a list with site url and list name. I want that in what ever siteurl and list available in this list, should populate a common dataset instead of adding of each site and append.
Logic behind that, today I have 4 sites, in fuure I have more sites, so I have to add/update the url in particular list. so I need to update report every time.
Can you please guid here that how can I read the site url from particular site url list and combine or append all the site url in one dataset.
Thanks again
Avian
try this one.
let
// Get the list of SharePoint sites and corresponding list names
Source = SharePoint.Tables("https://main.sharepoint.com", [ApiVersion = 15]),
SitesList = Source{[Name="SitesList"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(SitesList,{{"Site URL", type text}, {"List Name", type text}, {"Tenant Name", type text}, {"Site Code", type text}}),
// Add a custom function to fetch data from the corresponding SharePoint list for each SharePoint site
FetchList = (site as text, list as text) =>
let
Source = SharePoint.Tables(site, [ApiVersion = 15]),
List = Source{[Name=list]}[Content]
in
List,
// Add a new column "Custom" to call the custom function for each row
AppendLists = Table.AddColumn(#"Changed Type", "Custom", each FetchList([Site URL], [List Name])),
// Expand the tables from the "Custom" column into rows
#"Expanded Custom" = Table.ExpandTableColumn(AppendLists, "Custom", {"Employee ID", "Name", "Position"}, {"Employee ID", "Name", "Position"}),
// Select only the necessary columns
#"Removed Other Columns" = Table.SelectColumns(#"Expanded Custom",{"Tenant Name", "Site Code", "Employee ID", "Name", "Position"})
in
#"Removed Other Columns"
This script will:
- Fetch the list of SharePoint sites and their corresponding list names from your main SharePoint site.
- For each SharePoint site, it fetches the corresponding SharePoint list.
- It then expands each fetched SharePoint list into rows and appends these rows into the main table.
- Finally, it selects only the necessary columns.
Please replace "Employee ID", "Name", "Position" with your actual column names in the SharePoint Lists. If your lists have different structures, you'll need to modify the script accordingly to accommodate those differences.
Also, please replace "https://main.sharepoint.com" with your main SharePoint site URL and "SitesList" with your actual list name that contains the SharePoint site URLs and corresponding list names.
this is an experimental solution. you need to adjust it.
- avi0812653 years agoHelper III
Hello
Thanks for the solution. I am newbie in Power BI, how to use above mentioned in Power BI?
- rubayatyasmin3 years agoCommunity Champion
Happy to help. Kudos will be more appropriated.
Go to power query. Open advanced editor. And try copy paste the code. But you need to adjust the code value with your own value.
Did I help? If yes, hit 👍
If this is helpfull, accept this post as a solution.
- Anonymous2 years agoNot applicable
After editing your code with my own data, I was able to import multiple lists into Power BI. Howerver, I am stuck with refreshing the Dataset in Power BI Service because it does not support dynamic data source refreshing. I am aware of the solution for the Web.Contents function using RelativePath and Query headers but I am getting the authorization error when trying to use Web.Contents to fetch SharePoint lists even though I am admin of the SharePoint. Any suggestions on how to refresh dynamic data sources in Power BI Service when using SharePoint related function in Power Query?
- rubayatyasmin2 years agoCommunity Champion
make sure you have access to the data. also try following these articles,
Solved: Scheduled Refresh using Dynamic Data Source - Microsoft Fabric Community
Dynamic Data Source in Power BI / Blogs / Perficient
Solved: Dynamic data sources aren't refreshed in the Power... - Microsoft Fabric Community
- avi0812652 years agoHelper III
Hello Samper,
Can you please share your code, I am not able to fetch the lists.
Avian
- Anonymous2 years agoNot applicable
Sure here is my code:
let // Get the list of SharePoint sites and corresponding list names ParameterTable = Excel.Workbook(File.Contents("C:\myparameters.xlsx"), null, true), Sheet1_Sheet = ParameterTable{[Item="Sheet1",Kind="Sheet"]}[Data], #"Renamed Columns" = Table.RenameColumns(Sheet1_Sheet,{{"Column1", "Site URL"}}), #"Changed Type" = Table.TransformColumnTypes(#"Renamed Columns",{{"Site URL", type text}, {"Column2", type text}}), #"Renamed Columns1" = Table.RenameColumns(#"Changed Type",{{"Column2", "List Name"}}), // Add a custom function to fetch data from the corresponding SharePoint list for each SharePoint site FetchList = (site as text) => let Source = SharePoint.Tables(site, [ApiVersion = 15]) in Source, // Add a new column "Custom" to call the custom function for each row AppendLists = Table.AddColumn(#"Renamed Columns1", "Custom", each FetchList([Site URL])) in AppendListsand in the first comumn named "Site URL" of my parameter table, I have urls that look like this:
https://my.sharepoint.com/sites/AO1_FND
https://my.sharepoint.com/sites/AO1_FFF
and so on.Unfortunately, refreshing dynamic data sources is not possible in Power BI Service and if I use the function Web.Contents I face the authentication error even though I am the admin of the SharePoint and I can import the data using the code above. I am so lost and so puzzled that such an issue does not have a straighforward solution. Maybe uploading the data to Dataverse could solve this issue?
- jbrines1 year agoAdvocate III
rubayatyasmin Is it possible to just have it show all columns of the sharepoint list, if so what would the code be.
I tried your code but I was just getting an error
Expression.Error: The key didn't match any rows in the table.
Details:
Key=
Name=Site Sign In
Table=[Table]J
- rubayatyasmin1 year agoCommunity Champion
Hi jbrines thanks for reaching out. Can you tell me in which step you are getting the error? And yes it's possible to read the columns from SP list. Also, make sure you have replaced the necessary changes.
- jbrines1 year agoAdvocate III
Hi rubayatyasmin see screenshot below.
If I can get this working then I won't need the code for all columns
J