Forum Discussion
The Source has too many files
Hello SamyAbdul,
I will steer clear of creating a separate Lakehouse table for every file, but I wouldn’t cram everything into one giant wide table just to keep the table count down either. Microsoft’s medallion architecture guidance points the same way: keep the source data in Bronze, then refine it into structured, validated tables in Silver.
In Silver, I will group files that belong to the same business entity and have compatible schemas.
For schema changes, keep track of new columns and changes to existing ones so you can handle them before they affect downstream tables.
A single Bronze table with a JSON column is another option if you need to query the raw data centrally, but it isn’t a Fabric requirement.
One thing I will clarify first: are the 150 files different datasets, or are they different extracts of the same datasets? That makes a big difference to how I will structure the Lakehouse.
Docs
Schema evolution for Delta tables
Understand medallion architecture for Fabric with OneLake