Forum Discussion

jrauenbuehler's avatar
jrauenbuehler
Regular Visitor
2 years ago
Solved

Creating a Table Shortcut in Fabric from a Partitioned Delta Table

I have data in a Delta table that is stored in ADLS. It is partitioned by a column and therefore shows up in ADLS in a folder structure like `table_name/partition=partition_value/parquet_files` where...
data engineering
Anonymous's avatar
Anonymous
Not applicable
2 years ago

HI jrauenbuehler,

I think the shortcut should not be suitable to handle these transforming operations.
In my opinion, I'd like to suggest you use dataflow gen2 to get data from ADLS and apply transform operation to attach partitions to table columns.
In addition, you can also try to use notebook to load data to the dataframe and apply structure transform operations, write the modified dataframe back to the Lakehouse with delta format.

Regards,

Xiaoxin Sheng

  • jrauenbuehler's avatar
    jrauenbuehler
    Regular Visitor
    2 years ago

    @v-shex-msft But I'm able to create shortcuts from the same tables when I use a normal Lakehouse. I find it odd that I only run into this issue when I enable the Lakehouse Schemas to allow for additional schemas beyond just dbo. I read through the Lakehouse Schemas docs and it saw no indication that using this functionality would change anything in the way the shortcuts work beyond allowing custom schemas.