File data ingestion with materialized lake views (MLVs) in Microsoft Fabric is now available in preview. Instead of building separate ingestion pipelines to move raw files into tables, you can now ingest CSV and Parquet files directly from OneLake into a materialized lake view and make your bronze layer part of the same declarative pipeline as the rest of your medallion architecture.
This capability builds on the recent announcement of Materialized Lake Views in Microsoft Fabric (Generally Available), extending the simplified approach from table-based transformations to raw file ingestion.
Why this matters
Until now, you had to build and maintain a separate ingestion layer before you could get the benefits of MLVs, which had to start from tables. To get raw files into that first table, you'd reach for a shortcut transformation, a data pipeline, or a notebook load which is a separate step to build and operate before your declarative pipeline could even begin.
File data ingestion removes that step. Point a materialized lake view at a folder of files, and Fabric handles the ingestion, materializes the results as a Delta table, and plugs it straight into the same lineage, scheduling, data quality, and monitoring that power the rest of your medallion architecture. One programming model, from raw files to gold.
If you've referred to Simplifying Medallion Implementation with Materialized Lake Views, there was a missing link in the first mile: bronze, expressed declaratively. Now we have resolved that.
What is file data ingestion?
A file-ingesting materialized lake view is defined with a USING OneLake_Files clause instead of an AS SELECT query. You describe the source folder and format, and Fabric materializes it:
CREATE MATERIALIZED LAKE VIEW bronze.raw_orders
USING OneLake_Files
OPTIONS (
'format' = 'csv',
'path' = 'abfss://[email protected]/SalesLake.Lakehouse/Files/orders/',
'header' = 'true'
)
TBLPROPERTIES (
'schema_mode' = 'DYNAMIC',
'refresh_mode' = 'APPEND_ONLY'
);
That single statement gives you a queryable, refreshable bronze table. In preview, file data ingestion supports:
- CSV and Parquet source files.
- Recursive folder ingestion — files in nested subfolders are picked up automatically.
- Schema evolution with DYNAMIC mode, or a pinned schema with FIXED mode which gets auto detected during creation.
- Source traceability — every row carries a __filepath__ column recording that records its source file.
CSV parsing supports the header and delimiter options; additional CSV options and further capabilities are on the roadmap.
Build a complete medallion — From files to gold
Because a file-ingesting view behaves like any other materialized lake view once created, you can layer transformations on top of it and let Fabric orchestrate the whole chain in dependency order:
-- Bronze: ingest raw files directly from OneLake
CREATE MATERIALIZED LAKE VIEW bronze.raw_orders
USING OneLake_Files
OPTIONS ('format' = 'csv', 'path' = 'abfss://…/Files/orders/', 'header' = 'true')
TBLPROPERTIES ('schema_mode' = 'DYNAMIC', 'refresh_mode' = 'APPEND_ONLY');
-- Silver: clean and enforce data quality declaratively
CREATE MATERIALIZED LAKE VIEW silver.orders
(
CONSTRAINT valid_amount CHECK (amount > 0) ON MISMATCH DROP
)
AS SELECT id, customer, CAST(amount AS DECIMAL(10,2)) AS amount
FROM bronze.raw_orders;
-- Gold: aggregate for reporting
CREATE MATERIALIZED LAKE VIEW gold.daily_sales
AS SELECT customer, SUM(amount) AS total
FROM silver.orders
GROUP BY customer;
Fabric records the dependency from silver.orders → bronze.raw_orders → the source folder and refreshes them in order. When new files land, the change flows through silver to gold — no glue code, no orchestration to maintain.
Trace every file through lineage
File data ingestion is fully integrated into the materialized lake views lineage experience. The lineage graph starts from the source folder node, which feeds your bronze view and everything downstream:
- Select the source folder node to see the OneLake path and the files discovered in it.
- Select the file-ingesting view to inspect its format, schema mode, and refresh mode.
- Drill into any run to see how many files were processed and rows added — so you can confirm new data was picked up.
Pair that with the __filepath__ column to reconcile row counts back to individual source files:
SELECT __filepath__, COUNT(*) AS rows
FROM bronze.raw_orders
GROUP BY __filepath__;
Schema on your terms
File-ingesting materialized lake views support two schema modes, allowing you to choose between automatic schema evolution and a fixed schema contract:
- `DYNAMIC` (default): the view schema evolves with your data — new columns are added and dropped columns handled automatically across files and refreshes. Perfect for landing zones where source shape changes over time.
- `FIXED`: the schema is pinned at creation and drift is rejected — ideal when you need a stable contract for downstream consumers.
Refresh and scheduling, handled for you
Just like table-based materialized lake views, you don't orchestrate file ingestion from a notebook. Use lineage and scheduled refresh on the lakehouse to run views in dependency order and pick up new files automatically. Need to reprocess an entire folder on demand? A single full refresh does it:
REFRESH MATERIALIZED LAKE VIEW bronze.raw_orders FULL;
Key benefits
Together, these capabilities help simplify file-based ingestion and bring raw files into the same declarative workflow as the rest of your medallion architecture.
- One declarative model — from raw files to gold, no separate ingestion step to build or operate.
- Bronze in a single statement — replace bespoke COPY/pipeline/notebook loaders with USING OneLake_Files.
- End-to-end lineage — trace data from the source folder, through every layer, down to the individual file.
- Built-in data quality and scheduling — the same governance and orchestration you already rely on for table-based views.
- Schema flexibility — evolve automatically with DYNAMIC or lock a contract with FIXED.
Get started
- Open a lakehouse in Microsoft Fabric and drop your CSV or Parquet files into Files.
- Create a file-ingesting materialized lake view with USING OneLake_Files.
- Build silver and gold views on top, then schedule a refresh from the Materialized lake views → Manage experience.
Resources
- What are materialized lake views in Microsoft Fabric?
- Spark SQL reference for materialized lake views
- Manage materialized lake views lineage
- Tutorial: Build a medallion architecture with materialized lake views
Tell us what you think
We built file data ingestion to make the first mile of your medallion pipeline as effortless as the rest of it. Try it out on your own CSV and Parquet data and share your feedback on the Fabric Community — we can't wait to see the pipelines you build.
Microsoft Employee