Forum Discussion
Dataflows Gen2 cannot access Lakehouse Files — missing OneLake internal provider
Hello pbiadminjenhill,
What you’re seeing is expected behavior based on the current design of Dataflows Gen2 in Microsoft Fabric.
Dataflows Gen2 connects to a Lakehouse through the SQL analytics endpoint, which exposes Delta tables only. It does not surface the Lakehouse Files folder structure. As a result:
The Lakehouse connector shows tables, not /Files/...
File paths are treated as external sources and may prompt for a gateway
Combining Parquet files directly from Lakehouse Files isn’t supported
Incremental refresh applies to table sources, not raw file paths
This is a product capability boundary, not a tenant, region, trial, or GoDaddy limitation.
If you need to transform Parquet files stored in Files, first materialize them as Delta tables (via Notebook or Pipeline). Once the data exists as a Lakehouse table, Dataflows Gen2 can access it and support transformations and incremental refresh.
Hello pbiadminjenhill,
What you’re seeing is expected behavior based on the current design of Dataflows Gen2 in Microsoft Fabric.
Dataflows Gen2 connects to a Lakehouse through the SQL analytics endpoint, which exposes Delta tables only. It does not surface the Lakehouse Files folder structure. As a result:
The Lakehouse connector shows tables, not /Files/...
File paths are treated as external sources and may prompt for a gateway
Combining Parquet files directly from Lakehouse Files isn’t supported
Incremental refresh applies to table sources, not raw file paths
This is a product capability boundary, not a tenant, region, trial, or GoDaddy limitation.
If you need to transform Parquet files stored in Files, first materialize them as Delta tables (via Notebook or Pipeline). Once the data exists as a Lakehouse table, Dataflows Gen2 can access it and support transformations and incremental refresh.
Here is a screenshot of what I see - you can see I can go into my raw data but here it shows all the files together vs letting me choose one. My hope was to take this raw data, do some transformations like add a column for company name and then establish incremental refresh in the data flow. I can do it via pipeline as well and have done for now but feel like pipelines are more clunky and need more coding in order to do incremental refresh. Also, since I need to make some transformations like adding a column (I know I can add a custom column in pipelines as well) my thought was to use Data Flows Gen 2 to improve and cleanse the data so it can move from a raw zone to a cleansed data zone. Some online looking/Co pilot help is suggesting this is a Go Daddy limitation but would love feedback to see if anyone else has had this issue as well.
Hello pbiadminjenhill,
Thanks for the screenshot that behavior is consistent with how Dataflows Gen2 currently works in Microsoft Fabric.Although the UI lets you browse into Lakehouse → Files → Raw Data, Dataflows Gen2 still treats this as a generic file source, not a native Lakehouse Files source. That’s why:
All Parquet files in the folder are shown/combined together
You can’t select individual files or use folder structure as partitions
Incremental refresh isn’t available on this source
Gateway prompts can appear
The Lakehouse connector itself only surfaces Delta tables (via the SQL analytics endpoint)
So your approach of using Pipelines/Notebooks to add the company column and materialize the data as Delta tables in the Lakehouse is the correct first step. Once the data is in tables, Dataflows Gen2 is the right tool for cleansing, shaping, and configuring incremental refresh.
This isn’t a GoDaddy, tenant, trial, or region limitation — it’s the current product design boundary for Dataflows Gen2.