Forum Discussion
Lakehouse shortcut blob column data size limit?
I have a mirrored database that contains photographs from an Oracle database.
When accessing this data via the SQL Analytics endpoint of that mirrored database I have no issues.
I also got the schema of that table linked as a shortcut into a lakehouse in another workspace.
When accessing the binary column of the photos there, they are cut off after 16kB.
This is both true for the preview on the Fabric site, as well as connecting to the endpoint via a C# application, or SSMS.
I can't find anything that would confirm this as either a limitation or bug, or if there is a config I can adjust somewhere.
Hello sushiat
Microsoft documents that the Lakehouse SQL analytics endpoint exposes Delta Lake string and binary columns using fixed-size SQL types. Specifically, string and binary data from underlying Delta tables are represented as VARCHAR(8000) and VARBINARY(8000) in the SQL analytics endpoint, even when the source data is larger, which results in truncation when queried via T‑SQL or TDS-based clients such as SSMS.
This behavior is a documented limitation of the Lakehouse SQL analytics endpoint and applies regardless of whether the Delta table is native to the Lakehouse or exposed through a shortcut from another Fabric item. Microsoft distinguishes this endpoint from other SQL experiences in Fabric and explicitly notes that not all Delta data types or sizes are surfaced with full fidelity in the Lakehouse SQL layer.
Microsoft further clarifies that Fabric Warehouses and some other SQL experiences support larger data types, including VARCHAR(MAX) and VARBINARY(MAX) (preview), whereas the Lakehouse SQL analytics endpoint has not yet implemented equivalent support. As a result, large text or binary payloads that are intact in OneLake or Spark can appear truncated only when accessed through the Lakehouse SQL endpoint.
You'll find this mentioned here:
Limitations of Fabric Data Warehouse - Microsoft Fabric | Microsoft Learn
Workaround:
Microsoft-supported workarounds include accessing the data through Spark (notebooks or jobs), which reads Delta Lake data in OneLake without the SQL endpoint size limits, or copying/staging the data into a Fabric Warehouse where VARBINARY(MAX) is supported. Alternatively, where applicable, querying the mirrored database’s own SQL analytics endpoint avoids the Lakehouse SQL endpoint limitation, as mirrored databases are exposed through a separate SQL surface with different capabilities.
1 Reply
Hello sushiat
Microsoft documents that the Lakehouse SQL analytics endpoint exposes Delta Lake string and binary columns using fixed-size SQL types. Specifically, string and binary data from underlying Delta tables are represented as VARCHAR(8000) and VARBINARY(8000) in the SQL analytics endpoint, even when the source data is larger, which results in truncation when queried via T‑SQL or TDS-based clients such as SSMS.
This behavior is a documented limitation of the Lakehouse SQL analytics endpoint and applies regardless of whether the Delta table is native to the Lakehouse or exposed through a shortcut from another Fabric item. Microsoft distinguishes this endpoint from other SQL experiences in Fabric and explicitly notes that not all Delta data types or sizes are surfaced with full fidelity in the Lakehouse SQL layer.
Microsoft further clarifies that Fabric Warehouses and some other SQL experiences support larger data types, including VARCHAR(MAX) and VARBINARY(MAX) (preview), whereas the Lakehouse SQL analytics endpoint has not yet implemented equivalent support. As a result, large text or binary payloads that are intact in OneLake or Spark can appear truncated only when accessed through the Lakehouse SQL endpoint.
You'll find this mentioned here:
Limitations of Fabric Data Warehouse - Microsoft Fabric | Microsoft Learn
Workaround:
Microsoft-supported workarounds include accessing the data through Spark (notebooks or jobs), which reads Delta Lake data in OneLake without the SQL endpoint size limits, or copying/staging the data into a Fabric Warehouse where VARBINARY(MAX) is supported. Alternatively, where applicable, querying the mirrored database’s own SQL analytics endpoint avoids the Lakehouse SQL endpoint limitation, as mirrored databases are exposed through a separate SQL surface with different capabilities.