topic Re: Oracle Data to CSV / Parquet in lakehouse without creating a table in Data Engineering

Oracle Data to CSV / Parquet in lakehouse without creating a table

Noeleke1301 — Wed, 05 Jul 2023 12:13:45 GMT

Hi all,

I'm trying to make a layout for my lakehouse, based on the medaillon architecture. It states that my bronze layer should be stored in files.

However, when I load my Oracle data (on-prem database) into a dataflow and sink it into my lakehouse, it creates a table (but no file) which I don't want.

What's the best way to get my Oracle data and push it to my lakehouse as a csv file, without letting it create a table?

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

Noeleke1301 — Thu, 06 Jul 2023 06:01:22 GMT

Does no one have a solution for this?

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

ananthkrishna99 — Thu, 06 Jul 2023 17:25:40 GMT

Since you don't want to create it as a table, you can create a ADF Pipeline and load that data into ADLS Gen2 into required format and create a shortcut to that file in Lakehouse.

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

Noeleke1301 — Fri, 07 Jul 2023 06:36:39 GMT

Thanks for your reply.

So it's not possible to do this within Fabric?

I've read somewhere that Fabric Data Factory will have an on-prem connector in GA?

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

ananthkrishna99 — Fri, 07 Jul 2023 15:28:38 GMT

yes, that's true, please like the answer if you have found it helpful. Thank You!

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

GeethaT-MSFT — Tue, 11 Jul 2023 23:20:43 GMT

Hi, @Noeleke1301 Move data to the FILES section in Lakehouse, not the TABLES section. All data moved to TABLES section will be transformed to Delta Table automatically.

Regards

Geetha

Re: Oracle Data to CSV / Parquet in lakehouse without creating a table

Noeleke1301 — Wed, 12 Jul 2023 05:49:30 GMT

@GeethaT-MSFT When I select my lakehouse as a sink, I can only sink it as a table.

Also in my lakehouse, I can't move a table to the FILES section. I can only view the files behind it (but the files are not actually listed in the FILES section.