Gen2 Dataflow Started Failing - Nullable Issue

Hi Mrizzo21,

Welcome to the Fabric Community! It’s great to hear that you’ve found it helpful.🙂

The issue you're encountering with your Gen2 Dataflows sounds frustrating, especially since it seems to have arisen suddenly. The error message indicating that a source column is nullable while the destination column is not can often be linked to changes in the data itself or in the schema of the tables you're writing to.

Here are a few suggestions to troubleshoot and resolve the issue:

• Check Source Data: Even if you believe there are no nullable values in the source data, it’s worth double-checking for any new rows or changes in the Excel files that could introduce null values.
• Review Schema: Verify that the schema of your destination Fabric Warehouse tables has not changed. Ensure that the ID column and any other relevant columns are set correctly to handle the incoming data.
• Modify Query Steps: If there are any transformations or query steps that might inadvertently introduce null values, consider adding checks or filters to handle those cases explicitly before writing to the destination.
• Temporary Changes: As a test, you might try allowing nulls in the destination table for a specific run to see if that resolves the issue. This could help identify whether the problem is indeed related to nullability.
• Check for Recent Updates: Since the issue started recently, check if there have been any updates or changes to the Fabric platform or dataflow functionalities that could have introduced this behavior.

If the problem persists, it might be worth reaching out to the support team for additional insights, especially since it seems to have changed unexpectedly.

Good luck, and I hope you get this resolved soon!

Best regards,
Sai 
Data Engineer | 3x Microsoft Certified

If this helps, kindly mark this as the accepted solution! 👍