Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
smpa01
Super User
Super User

Why Dataflow does not load new columns from SQL

I have a dataflow querying a SQL table, such as

 

 

seelct * from [dbo].[tbl]

 

 

 

IF the dataflow is created today and a column called [colA] gets added to SQL table tomorrow ([colB] day after tomorrow), unless I manually refresh the dataflow tomorrow (and the day after tomorrow), the dataflow will not load the newly added columns automatically [colA] tomorrow ([colB] day after tomorrow)  in the load.

 

I thought scheduled refresh would load everything but it does not unless I open the df and manually refresh the df.

 

What can I do to automatically load the newly added SQL columns as and when they get added without me requiring any manual intervention.

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
4 REPLIES 4
v-janeyg-msft
Community Support
Community Support

Hi, @smpa01 

 

I tested two refresh methods, but I found no change in the dataflow view of the service.

vjaneygmsft_1-1642665538964.png

New columns will be updated only when you save the editor table after loading.

vjaneygmsft_0-1642665514627.png

vjaneygmsft_2-1642665749514.png

I think it should be by design.

 

Best Regards
Janey Guo

@v-janeyg-msft thanks for testing it out. Is there any chance for the default behavior to inlude newly added colums in the near future without manual intervention required?

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs

Hi, @smpa01 

 

This is not easy to say. Refresh in pbi desktop and pbi service is different, in service only existing columns are refreshed. If you delete a column and then refresh, you will find that it will report an error. You can only save the latest columns after reloading in the editor.

 

If this post helps, please consider Accept it as the solution to help the other members find it more quickly.

 

Best Regards,
Community Support Team _ Janey

AlexisOlson
Super User
Super User

I'm pretty sure SELECT * gets turned into SELECT [list of columns] internally, so you need to re-run the dataflow to recreate the list of columns.

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.