Cannot Load All Table to Lakehouse After Transformation on Dataflow

Hi BeyzaKzlky 

Thanks for using Microsoft Fabric Community.

Apologies for the issue that you are facing here.

Here are the possible reasons for missing columns in your published Dataflow Gen2 table:

Incomplete Schema Inference:

• Check for Unsupported Data Types: Dataflow might skip columns with unsupported data types. Ensure all columns have compatible types (e.g., string, number, date/time).
• Enable Explicit Schema Definition: If automatic inference is unreliable, explicitly define the schema in the source dataflow or sink settings.

Transformation Issues:

• Review Transformation Steps: Examine each step for filtering, projections, or aggregations that might inadvertently remove columns.
• Verify Joins: Ensure all necessary columns are included in the output of joins.

Output Sink Configuration:

• Inspect Output Settings: Double-check the sink configuration to guarantee that all desired columns are mapped to the destination table.
• Confirm Column Mapping: Verify that columns are correctly mapped in the output settings, especially after schema changes.

I hope this information helps. Please do let us know if you have any further questions.