Forum Discussion

smpa01's avatar
smpa01
Community Champion
4 years ago

Why Dataflow does not load new columns from SQL

I have a dataflow querying a SQL table, such as

 

 

seelct * from [dbo].[tbl]

 

 

 

IF the dataflow is created today and a column called [colA] gets added to SQL table tomorrow ([colB] day after tomorrow), unless I manually refresh the dataflow tomorrow (and the day after tomorrow), the dataflow will not load the newly added columns automatically [colA] tomorrow ([colB] day after tomorrow)  in the load.

 

I thought scheduled refresh would load everything but it does not unless I open the df and manually refresh the df.

 

What can I do to automatically load the newly added SQL columns as and when they get added without me requiring any manual intervention.

Dataflow
Need Help

4 Replies

  • AlexisOlson's avatar
    AlexisOlson
    Super User
    4 years ago

    I'm pretty sure SELECT * gets turned into SELECT [list of columns] internally, so you need to re-run the dataflow to recreate the list of columns.

  • v-janeyg-msft's avatar
    v-janeyg-msft
    Community Support
    4 years ago

    Hi, smpa01 

     

    I tested two refresh methods, but I found no change in the dataflow view of the service.

    New columns will be updated only when you save the editor table after loading.

    I think it should be by design.

     

    Best Regards
    Janey Guo
    • smpa01's avatar
      smpa01
      Community Champion
      4 years ago

      v-janeyg-msft thanks for testing it out. Is there any chance for the default behavior to inlude newly added colums in the near future without manual intervention required?

      • v-janeyg-msft's avatar
        v-janeyg-msft
        Community Support
        4 years ago

        Hi, smpa01 

         

        This is not easy to say. Refresh in pbi desktop and pbi service is different, in service only existing columns are refreshed. If you delete a column and then refresh, you will find that it will report an error. You can only save the latest columns after reloading in the editor.

         

        If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

         

        Best Regards,
        Community Support Team _ Janey