Why Dataflow does not load new columns from SQL

[smpa01]

I have a dataflow querying a SQL table, such as

seelct * from [dbo].[tbl]

IF the dataflow is created today and a column called [colA] gets added to SQL table tomorrow ([colB] day after tomorrow), unless I manually refresh the dataflow tomorrow (and the day after tomorrow), the dataflow will not load the newly added columns automatically [colA] tomorrow ([colB] day after tomorrow)  in the load.

I thought scheduled refresh would load everything but it does not unless I open the df and manually refresh the df.

What can I do to automatically load the newly added SQL columns as and when they get added without me requiring any manual intervention.

[v-janeyg-msft]

Hi, @smpa01 

I tested two refresh methods, but I found no change in the dataflow view of the service.

New columns will be updated only when you save the editor table after loading.

I think it should be by design.

Best Regards

Janey Guo

[smpa01]

@v-janeyg-msft thanks for testing it out. Is there any chance for the default behavior to inlude newly added colums in the near future without manual intervention required?

[v-janeyg-msft]

Hi, @smpa01 

This is not easy to say. Refresh in pbi desktop and pbi service is different, in service only existing columns are refreshed. If you delete a column and then refresh, you will find that it will report an error. You can only save the latest columns after reloading in the editor.

If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

Best Regards,
Community Support Team _ Janey

[AlexisOlson]

I'm pretty sure SELECT * gets turned into SELECT [list of columns] internally, so you need to re-run the dataflow to recreate the list of columns.