Forum Discussion
Oracle Source to Lakehouse
Hi trebllaw,
Perhaps you can try to use shortcut feature to getting data. It will create a shortcut to load data from remote data source without data movement.
Since current not support direct create oracle shortcut, you may need to mapping oracle data to supported data source(e.g. Azure, Dataverse…), then you can use shortcut to getting data from azure.
Unify data sources with OneLake shortcuts - Microsoft Fabric | Microsoft Learn
Regards,
Xiaoxin Sheng
As much as possible, we'd like to avoid additional costs from using Azure since we are already paying for 3 P3 capacities as of the moment. That is why we didnt go the use staging route as that would consume Data Lake Gen 2 storage, which is an additional cost.
HI trebllaw,
Perhaps you can consider using sempy link which can used to getting data from Semantic model.
What is semantic link? - Microsoft Fabric | Microsoft Learn
You can design a PBI report on desktop with live connection model to the oracle data source and publish it to power bi service workspace that assigned with fabric capacity.
After you assign gateway to handle its refresh, you can use notebook to getting data from that model and loading to Lakehouse.Semantic link and Power BI connectivity - Microsoft Fabric | Microsoft Learn
Regards,
Xiaoxin Sheng
Unfortunately, this wouldnt work for us since we are going to be loading huge amount of data from multiple FACT tables.
The main reason we are trying to use pipelines is that it is advertised to be the ideal option to move large amounts of data, being able to load petabytes of data pretty quick, but based from our tests, we are getting about 5Kb/s of throughput. For 3.5, we only managed to process and transfer about 1.5M rows. We may be doing something wrong here we are not sure.
Here are some of the things that might be of note:
1. Oracle source system isnt partitioned.
2. We've set the pipeline to dynamic partitioning and set one of the key fields as index.
3. Parallel copies automatically set to 3
