Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Join us at FabCon Vienna from September 15-18, 2025, for the ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM. Get registered

Reply
FabricNZ
Helper I
Helper I

Gen2 DataFlow inserts null values into Warehouse

HiYa,

I have two tables in the same fabric warehouse, Staging and FACT. The Gen2 transformation moves all data from staging to fact and creates a single custom column along the way called x_isCostTagged.Screen Shot 2025-01-11 at 9.37.10 PM.png

This is a simple value of 1. It is of type whole number and the target schema has a corresponding x_isCostTagged field of type INT. The problem is the transformation runs, all columns are populated except the custom field which is loaded with null values. Screen Shot 2025-01-11 at 9.42.34 PM.png

Am I being batty in that this should just work? I have tried it with text fields and varchar, etc. Nothing seems to get inserted. Thank you so much for your help.

1 ACCEPTED SOLUTION

Hi @FabricNZ,

 

Thank you for reaching out to the Microsoft Fabric Community Forum.

 

After reviewing the issue of Dataflow Gen2 inserting null values, I reproduced your exact scenario and used custom column to create value 1 column.


I tried both replace and append, the data has successfully reached the destination table. Try to check if you have made any errors in dataflow gen2 and retry it.

 

For reference, please go through the below screenshots:

 

First, open your workspace in your Fabric account, then select Warehouse.

vkpolojumsft_0-1737045876774.png

 

Next, open the warehouse, select the 'get data' option, and then click on 'new dataflow gen 2'.

vkpolojumsft_1-1737045921074.png

 

After clicking on the Dataflow Gen2, select the 'Get Data' option. It will prompt you to provide sample data through Excel.

vkpolojumsft_2-1737045964843.png

 

After importing the data and adding the custom column to the table using the custom column option, select the destination at the bottom of the page and click on the warehouse.

vkpolojumsft_3-1737046022553.png

 

Please use the same warehouse, name the table Facttable2, select warehouse(kannawh), and then click next.

vkpolojumsft_4-1737046059903.png

 

Please select the Append option, then click Save Settings, and publish it.

vkpolojumsft_5-1737046107016.png

 

After opening the warehouse and refreshing the table, please refer to the output in the image below.

vkpolojumsft_6-1737046135128.png

 

If you have any further queries, please feel free to reach out to us we will guide you accordingly. If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

 

Thank you.

 

 

View solution in original post

3 REPLIES 3
FabricNZ
Helper I
Helper I

Thank you for your detailed reply.  The silly thing just started working again the next day so I'm chalking it up to just one of those things! 

FabricNZ
Helper I
Helper I

I forgot to add, sorry that when I say nothng gets added I am querying the Warehouse directly and not through PowerBI or any semantic model.  This appears to be a simple failure of not being able to insert the value of a custom column directly into the fabric warehouse. I haven't got a complex custom column either. I would expect a column full of the number 1. 

 

Interestingly with the two presented modes (append or replace), with append, I do not get to see the column I created in the custom mapping. With replace (which I do not wish to use), the custom column shows up and I can map it and its type to the destination.  With replace, the column comes across and the data flows through into the warehouse. So, my simple transfer works it works on replace table, but not append. ANy ideas folks?Screen Shot 2025-01-12 at 11.50.56 AM.png

Additional note: Now that I had selected replace table and moved back to append, the column showed up in the mapping. It always existed in the destination schema. Sheesh....

Hi @FabricNZ,

 

Thank you for reaching out to the Microsoft Fabric Community Forum.

 

After reviewing the issue of Dataflow Gen2 inserting null values, I reproduced your exact scenario and used custom column to create value 1 column.


I tried both replace and append, the data has successfully reached the destination table. Try to check if you have made any errors in dataflow gen2 and retry it.

 

For reference, please go through the below screenshots:

 

First, open your workspace in your Fabric account, then select Warehouse.

vkpolojumsft_0-1737045876774.png

 

Next, open the warehouse, select the 'get data' option, and then click on 'new dataflow gen 2'.

vkpolojumsft_1-1737045921074.png

 

After clicking on the Dataflow Gen2, select the 'Get Data' option. It will prompt you to provide sample data through Excel.

vkpolojumsft_2-1737045964843.png

 

After importing the data and adding the custom column to the table using the custom column option, select the destination at the bottom of the page and click on the warehouse.

vkpolojumsft_3-1737046022553.png

 

Please use the same warehouse, name the table Facttable2, select warehouse(kannawh), and then click next.

vkpolojumsft_4-1737046059903.png

 

Please select the Append option, then click Save Settings, and publish it.

vkpolojumsft_5-1737046107016.png

 

After opening the warehouse and refreshing the table, please refer to the output in the image below.

vkpolojumsft_6-1737046135128.png

 

If you have any further queries, please feel free to reach out to us we will guide you accordingly. If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

 

Thank you.

 

 

Helpful resources

Announcements
Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.

June 2025 community update carousel

Fabric Community Update - June 2025

Find out what's new and trending in the Fabric community.