pulling new columns from SharePoint List

[CROKat]

I have an existing Power BI tool that is pulling data from a SharePoint List on a daily basis. My team added two new columns to the List, and I need those two new columns to also pull into the Power BI tool. How would I do that? I thought they would come in automatically and I don't see where to add them in table view.

[v-jtian-msft]

Hi,uzuntasgokberk ,thanks for your concern about this issue.

Your answer is excellent!
And I would like to share some additional solutions below.
Hello,@CROKat .I am glad to help you.

When you add two new columns to your sharepoint list, the data source has changed schema, this change is not supported by the refresh function in Power BI Service, you need to perform the refresh function in Power BI Desktop and re-publish the designed report to Service.

For schema changes (new columns or deleted columns), you cannot get the new schema data by re-get data. You need to click the Refre option in Power BI Desktop to update your model.
Normally, the system will automatically add the new columns to the model.
Here is my test, I hope it will be helpful to you

I simulated how you described how the two new columns should be fetched in power BI Desktop when two new columns are added to the sharepoint list data source (by clicking on the Refresh option)

In Power Query (transform data in Power BI Desktop)

In Power BI Desktop

This is my test:

I create a new List in SharePoint

I create  three columns and add data

I then changed the Online List I had just created in the SharePoint site, added two columns of data to it, modified its schema, and re-fetched the data from Power BI Desktop

When I  refresh in Power BI Query, the schema is updated and the data for the new columns is loaded in

I found other issues with similar problems, some users mentioned that after clicking refresh, the new columns were still not displayed, they finally got the new columns by checking the conversion steps in the power query (the new columns may be hidden by the system).

To summarize, the actions you need to perform include the following steps:

1.Refresh the data source:
Open Power BI Desktop and load your report.
Click the Refresh button in the Home tab. This will reload the data source and may automatically include the newly added columns.
2.Edit Query: In Power BI Desktop, click on the Home tab and select Edit Query.
In the Query Editor, locate your SharePoint data source.
Ensure that the table containing the new columns is selected in the Navigation step.
If necessary, click the Expand Columns button to ensure that the new column is selected and included in the data model.
Advanced Editor:
3.If the new column is still not visible after refreshing and editing the query, you can manually add the new column using the Advanced Editor.
In the Query Editor, click Advanced Editor.
Check and modify the M-code to ensure that the new column is correctly included in the query. For example, make sure that the name of the new column is included in the Table.SelectColumns function.
Delete the old columns and re-add them:
Sometimes Power BI still uses the old internal column names after the column names in the SharePoint list have changed. You can try deleting the old columns and re-adding the new columns to ensure that the new column names are recognized correctly.
After completing the above steps in Power BI Desktop, save and republish the report to the Power BI service.

URL:

How to Fix the “List” Issue in PowerBI when Using SharePoint People Picker Fields | @cdsmythe
Solved: New Column from Sharepoint list will not show up i... - Microsoft Fabric Community
Solved: New Column in Sharepoint Does Not Show Up in Power... - Microsoft Fabric Community
Solved: Getting a newly added column in SharePoint List to... - Microsoft Fabric Community

I hope my suggestions give you good ideas, if you have any more questions, please clarify in a follow-up reply.
Best Regards,
Carson Jian,
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[uzuntasgokberk]

Hello @CROKat ,

Open your existed power bı report in pbı desktop -> go to transform data(power query) -> you might be selected required columns that's why new columns is now shown. Therefore added another required columns in your existed step or add steps.

Best Regards,
Gökberk Uzuntaş

LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/

Medium: https://medium.com/@uzuntasgokberk

İf this post helps, then please consider Accept it as solution and kudos to help the other members find it more quickly.

[shreebidwai]

For this explanation, I’m assuming you are using a SharePoint list as the data source in Power BI and have configured it as a component.

When you connect to a SharePoint list, Power BI imports both the data columns from the list and the associated SharePoint metadata columns. To work only with specific data columns, you may have applied a filter to select the necessary columns and exclude the rest. As a result, when new columns are added to the SharePoint list, Power BI does not automatically include them because it retains your previous selection of columns.

How to Fix the Issue:

1. Go to the Edit Queries section in Power BI:

   • Locate the table connected to the SharePoint list.
   • Right-click on the table and select Edit Query.

2. In the query editor, find the Applied Steps section on the right-hand side.

   1. 

3. Look for steps such as "Remove other columns" or "Remove columns":

   • Click the gear icon next to these steps to open the column selection settings.
   • 

4. In the column selection dialog, you’ll see a list of all available columns with checkboxes.

   • Find the newly added columns in the SharePoint list.
   • Check the boxes next to these columns.
   • 

5. Once done, click OK, then close and apply the changes.

This process ensures that the new columns are included in your dataset.

I hope this resolves your issue! If it does, kindly mark this as the answer. Let me know if you have further questions!