Forum Discussion
Dataflow - How can I stop automatic steps being added
I have a Datalake with multiple folders with lots of blobs that are JSON files each folder has different json structures and some are lists and others are records. I am trying to use a single dataflo...
I may have found a way around the one that shows a fX (function) symbol and says "Remove Columns" ...
Table.RemoveColumns(#"Previous step name", Table.ColumnsOfType(#"Removed columns 1", {type table, type record, type list, type nullable binary, type binary, type function}))
The PowerQuery editor for Dataflows seems to be adding this in an attempt to get rid of columns where every current value is null. However, I want it to retain the column because I am writing the code ahead of the data - as in the system doesn't yet have values in it yet, but will in the future.
Try explicitly setting the type of the columns that you want to keep. My best guess is that it was implicitly converting my column to the nullable binary. But by explicitly setting the type in the previous step, it finally kept my column. It was a little funky in my case because that column was explicitly set in the original query which was merged with my final table, so not sure why I had to tell PowerQuery the column type twice.
The PowerQuery editor for Dataflows seems to be adding this in an attempt to get rid of columns where every current value is null. However, I want it to retain the column because I am writing the code ahead of the data - as in the system doesn't yet have values in it yet, but will in the future.
Try explicitly setting the type of the columns that you want to keep. My best guess is that it was implicitly converting my column to the nullable binary. But by explicitly setting the type in the previous step, it finally kept my column. It was a little funky in my case because that column was explicitly set in the original query which was merged with my final table, so not sure why I had to tell PowerQuery the column type twice.
Could you provide more details on this solution? I can't figure out how to implement it.
- Select column(s) in the editor
- From the Transform menu, click the Data Type drop down button and select the desired data type
- Repeat for each column
NOTE: For step #1, you can Ctrl+Click to select multiple non-contiguous columns or Shift+Click to select multiple columns if you want to set several to the same data type at once, e.g. if you have 5 columns that are integers and 2 that are text and 3 that are decimal, you would select the 5 integers and transform them, then select the 2 text and transform them, then the 3 decimal
This right here, worked like a charm!
