Forum Discussion

S4CSurrey's avatar
S4CSurrey
Icon for Helper I rankHelper I
6 years ago
Solved

New column added to Dataflow not displaying in Dataset

I have a pre-existing Dataflow that is calling data over REST. I have a Dataset created for this Dataflow, and a connected Report displaying data correctly.

 

Today, I edited the query in the Dataflow to expand out a new piece of data, a basic string type column. I refreshed the data, and the Power Query Editor is now displaying my table of data with the new column, and valid entries in the rows of this new column. 
After this was done, I saved in Power Query Editor, and no errors came up. I refreshed my Dataflow, and once completed, I refreshed the Dataset.

 

Opening the Dataset in Excel for analysis, I cannot see my my column from the Dataflow's data query.

 

What might be causing this. Any help is appreciated.

dataflow
  • v-lid-msft's avatar
    v-lid-msft
    6 years ago

    Hi S4CSurrey ,

     

    Could you please check if you can see the new column when using the same query in Power BI Desktop?


    Best regards,

     

12 Replies

  • chaz2jerry's avatar
    chaz2jerry
    Icon for Advocate IV rankAdvocate IV
    5 years ago

    I think I have a similar issue and found the root cause.  In my scenario, I created a custom column in Decimal format.  Preview data in dataflow looks good, but the new column does not show up in the entity/table after refresh (as confirmed by connecting to this Dataflow table from PBI Desktop).  I had this issue a couple months back, and could not confirm why it was happening.  Then today I looked into the Advanced Editor (M Query) and found that the final "in" statement points to not the last query step as usual, but points to the step before I made the latest custom column.  Example of the M Query having the issue.  I assume this issue will be fixed when I adjust the M Query manually, not sure why it occurred in the first place.  

     

      #"step 6" = Table.TransformColumnTypes(#"Changed Type", {"SOLD_TO", type text}),
      #"step 7" = Table.AddColumn(#"Changed Type1", "Sales incl Tariff", each [Sales] + [TARRIF]),
      #"step 8" = Table.TransformColumnTypes(#"step 7", {{"Sales incl Tariff", Currency.Type}})
    in
      #"step 6"
  • v-lid-msft's avatar
    v-lid-msft
    Icon for Community Support rankCommunity Support
    6 years ago

    Hi S4CSurrey ,

     

    Could you please check if you can see the new column in a blank report based on the dataset? What is the data type of this new column , based on this document: A dataflow must output one of the following types: Date/Time, Decimal Number, Text, Whole number, Date/Time/Zone, True/False, Date, Time

     


    Best regards,

     

    • S4CSurrey's avatar
      S4CSurrey
      Icon for Helper I rankHelper I
      6 years ago

      I tried in a new report, not there still.

       

      In Power Query, it is currently a Text type column. The source data is just a String type from my online database. Other string type fields are coming in and set to be a Text type column, and are working.

       

      As a test, I set the column type to be Decimal, and the column's data still displayed properly. Switched back to Text, refreshed the data, and the column's data still displayed properly. Refreshed the Dataflow and Dataset, re-checked the new test report and analyzed it in Excel, and the column of data is still not accessible.

      • v-lid-msft's avatar
        v-lid-msft
        Icon for Community Support rankCommunity Support
        6 years ago

        Hi S4CSurrey ,

         

        Could you please check if you can see the new column when using the same query in Power BI Desktop?


        Best regards,

         

  • retailbusiness's avatar
    retailbusiness
    Frequent Visitor
    4 years ago

    the same issue here, I have to open the dataset on Desktop PBI and refresh from table "refresh data" option to display new columns from dataflow. Online dataset refresh won't update table schema! I don't know if it's normal behavior.

  • denpries's avatar
    denpries
    Icon for Resolver I rankResolver I
    3 years ago

    I have an exact similar issue to the one you describe.

    I have added in a dataflow 3 new columns.

    Aother, existing dataflow, combines this dataflow with another flow.

     

    In the flow editor on the service you see the three new columns as last step. When refreshing, all goes well.

    PBI desktop however cant see the new 3 columns, also not when you clear cache.

     

    The only resolution is to e.g. add a useless replace operation in the query of the flow on an arbitrary column, not per se the 3 missing ones, 'e.g. replace X with identical X' . After refreshing the new columns also appear in powerbi desktop.

    Strangely enough, when removing the useless replace again, the columns also stay.

     

    What i think, is that something in the underlying scheme is not updated, until you force it to. (or something like that)