Dynamic Schema limitation in DataFlow Gen2

Idea: To automate the process of Dynamic Schema ingestion to a Data Destination using DataFlow Gen2.

We are working on an Excel file located at SharePoint Folder which is having Dynamic Schema with a column addition every week.

When we are trying to ingest the SharePoint Dynamic data to a Lakehouse using Dataflow Gen2 with Data destination option as Replace and Dynamic Schema, the DataType of new column isn't predicted accurately and in result the Lakehouse table doesn't get updated with new column automatically. Manual intervention is always required in this activity and corresponding pipeline is impacted.

We can automate this process where the new columns added to a SharePoint Excel file can be ingested to the given Data Destination using DataFlow Gen2 with option as Replace and Dynamic Schema without manually updating the DataType of new column every time before DataFlow Gen2 refresh.

Thanks to acknowledge and work on it.