Forum Discussion
Dataflow Gen2 write no value in Lakehouse
Theo_MS, I have the exact experience you are having, and it started about the same time you raised this post. I have a running theory I have tested on my end for a solution, but if you could confirm if it works for you that would be great.
My Dataflow is a regular CSV to Lakehouse flow. For simplicity, I have two columns to load,
Here is the change I have noticed since I started experiencing this error. When using the Automatic Settings in the Lakehouse Data Destination screen, the "Destination Columns" no longer add an underscore if the column names have a space. It will stick to the column names with a space in them.
This results in NULL values in my entire table
My Current Solution: I rename the columns to add an underscore where needed so as not to have any spaces in the column name. I kept the Lakehouse settings the same.
After the refresh was done, the values then appeared.
I guess that this has to do something with Lakehouses columns and spaces; seems like it's causing some issues with our dataflow. I hope this helps troubleshoot your issue.
When we have hundreds of columns, this is not a feasible solution.
Hello AlfredOne - in the interim, here is a code you can use to automatically update all column headers in a table to replace the spaces with underscores.
let
Source = YourSourceHere, // Replace with your actual data source
RenameColumns = Table.TransformColumnNames(Source, each Text.Replace(_, " ", "_"))
in
RenameColumns