All Columns are not getting selected from a table until Preview is refreshed

[PBI__User]

Hi All,
I’m getting a "missing column" error when i try to refresh a PBIX as the navigation step is not loading all the columns from a table during the initial refresh, due to which the later transformations are failing as the column name mentioned in the transformation are not loaded to power BI. But when I click refresh preview it brings back all columns and the error in later transformation disappears. Could anyone help here?

Also, I face this issue only while refreshing against a read only cluster in databricks.

Below are some of the options which I thought might work but it did not work in our case.

1. Clear cache and clear permissions

2. Used "Removed Other Columns/Select Columns" rather than "Remove columns" transformation.

Thanks in Advance,

[Greg_Deckler]

@PBI__User Hard to know specifically, sounds like a transformation drops the column but the column is referenced in a later step. Can you post the M code from Advanced Editor and name the offending column?

[PBI__User]

Hi @Greg_Deckler ,

Thanks for the response. Given below the steps in my power query. Could you please take a look?

1) Main query fetches all the tables in the selected schema (server details and schema name is passed as a parameter in this query)

2) All other queries/tables use the main query as source.

When I refresh, I get an error "The column Rank1 of the table was not found" as the rename step is referencing this column name. When I checked the previous step "DataTable" I couldn't see the column Rank1 in the result set though this column exists in the backend table. When I refresh preview "Rank1" column gets loaded and the missing column error disappears. This issue is not always on the same table, but occurs on different tables each time I refresh.

Thanks in advance,