New column in source table not appearing in dataset after refresh?

[cparker4486]

I found this post but it doesn't play out for me. http://community.powerbi.com/t5/Service/New-column-in-source-table-not-appearing-in-dataset-after/m-...

The only way I've been able to get new columns to appear is to change the Source line in the Advanced Editor. Specifically, I have to change the Columns value to reflect the current amount.

Source = Csv.Document(File.Contents("C:\source\file.csv"),[Delimiter=",", Columns=12, Encoding=1252, QuoteStyle=QuoteStyle.None]),

Is this the expected behavior or is it specific to CSV source type?

[v-qiuyu-msft · Accepted solution]

Hi @cparker4486,

I will report this issue internally, and keep you update once I get any feedback.

Best Regards,

Qiuyun Yu

[karpandi]

As I haven't seen this solution above, I add it here.
Under Applied Steps > Removed Other Columns it is possible to tick the missing newly added column.

[melly]

Thank you for this post.  Very helpful.  

[obera]

Hi folks,

I am having this issue but I do not see a place in my code that specifies number of columns to change/delete it, and it is still not showing my additional 3 columns.

let
Source = Folder.Files("C:\FILEPATHHERE"),
#"Filtered Hidden Files1" = Table.SelectRows(Source, each [Attributes]?[Hidden]? <> true),
#"Invoke Custom Function1" = Table.AddColumn(#"Filtered Hidden Files1", "Transform File from Assignment Group Duration", each #"Transform File from Assignment Group Duration"([Content])),
#"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
#"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File from Assignment Group Duration"}),
#"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File from Assignment Group Duration", Table.ColumnNames(#"Transform File from Assignment Group Duration"(#"Sample File (2)"))),

[JonasWesth]

When Query a folder with CSV. files, you can get the same problem, but you have to apply the solution to the supporting query under "Transform File from (Query Name)". Here you select the query used for the fily taped marked with the table icon, and then open the advanced editor and apply above solution. I just added one to the amount of columns and worked as a charm. 

[DevDelwyn]

Same problem as Obera in that when using 'Source = Folder.Files" there is no 'Csv.Document' command to indicate the number of columns.

Any guidance or suggestions would be very welcome.

[griffian]

I had the same issue and could not find where to change the number of columns being picked up.

I found it in the Advanced Editor of the 'Transform Sample File.....' under 'Sample Query' and not in the main query feeding my table.

Once I had increased the number of columns and refreshed, the additional columns appeared.

[nickjtay1 · Accepted solution]

I spoke with Microsoft Support and have a fix.

PBI Desktop doesn't recognize the new column for March that was added:

When you open the advanced editor, you will see that the original number of columns is defined:

Remove the definition of the number of columns:

When you click "Done" and return to the query editor the new column appears:

[Manreaux]

Thank you so much for this. I'm sure I'm not supposed to comment simply to thank someone, but in this instance, where you saved me so completely, I will spit in that rule's face. I'm very thankful for this answer.

[mnayab]

It was very helpful. Thanks.

[Anonymous]

One of the best answers ever I have come across. Thanks 🙂

It really needed lot of searching. @nickjtay1 

[ScottFerguson]

I ran into this problem and found that referencing the data as a table solved the problem. I have since adopted importing excel/ csv source data as a table as a best practice.

[ofariaga]

Hi scott, what do you mean by "referencing the data as a table"? Is it some kind of different way of importing the CSV file or is it a transformation after importing?

[about_blank]

The problems are still there. (Power BI Desktop)

[zebing2]

Did this ever got resolved? Not being able to refresh my data sources quickly when I am frequently changing columns is a major problem. My csv files aren't refreshing properly and I'm wondering if I need to go back to Tableau to get things done. 

Is there a fix for this yet?

[v-qiuyu-msft]

Hi @cparker4486,

Based on my test, when we connect to CSV file in Power BI Desktop, the added column in a CSV file will not update whether we click Refresh button in Home page or Refresh Preview button in Query Editor. As you mentioned, we need to modify Columns value then the added column will display. In my opinion, Columns parameter limits the number of returned column from CSV file, if we added columns in CSV file, Columns parameter value needs to be modified as well to make new column display.

Also I have tested with SQL Server database, if we haven't written query to indicator which columns need to be returned(eg: select column1, column2 from table1), the new column will update after clicking Refresh or Refresh Preview button. I thinks the issue should be related to CSV data source itself, as each data source are defined differently in Power Query. You can take a look at Advanced Editor in Query Editor.

If you have any question, please feel free to ask.

Best Regrads,

Qiuyun Yu

[cparker4486]

Hi @v-qiuyu-msft,

Then is this considered a bug and will be addressed at some point?

Thanks.

[v-qiuyu-msft · Accepted solution]

Hi @cparker4486,

I will report this issue internally, and keep you update once I get any feedback.

Best Regards,

Qiuyun Yu

[B_Real]

Any update on this? It's still an issue.

[Anonymous]

go to Advanced Editor:

remove 'Columns=3, '  (probably you have a different number of columns)

click 'Refresh all'

[Anonymous]

For me it is not taking the new columns so I changed the Columns= 8 in the advance editor according to the columns I need from csv file. Then automatically new columns got loaded into Power BI