Forum Discussion

libpekin's avatar
libpekin
Icon for Advocate II rankAdvocate II
1 year ago

Issue Ingesting Delta Sharing Tables with Deletion Vectors in Fabric Lakehouse Notebooks

1. Background: I'm building an end‑to‑end ingestion pipeline entirely within a Microsoft Fabric Lakehouse Notebook. My source is an AWS‑hosted Delta Share, and target is OneLake staging (schema raw)...
data engineering
Notebook
BhaveshPatel's avatar
BhaveshPatel
Icon for Super User rankSuper User
1 year ago

Try single delta table at a time. It is working in my scenario. Deletion Vectors..It is more advanced version of OPTIMIZE ( Delta Lake Deletion Vectors | Delta Lake ). Also, Try creating a delta table first...it becomes easy then..

 

libpekin's avatar
libpekin
Icon for Advocate II rankAdvocate II
1 year ago

BhaveshPatel 

Thank you for your response. Just to clarify—the question isn’t about configuring Deletion Vectors in Fabric. Rather, we’re seeking viable workarounds within Fabric when ingesting Delta Share tables that we don’t control, particularly when the provider has enabled Deletion Vectors and is unwilling to modify the table properties.

What strategies exist to read or ingest from such sources given current Fabric platform limitations?