Data Model Size - Import from Oracle DB vs. Data Flow

[Dave1mo1]

Hello,

I have a semantic model that populates my model entirely from Gen 1 dataflows - there are 17-18M rows in these fact tables. The dataflows point at SharePoint folders that contain CSV files. That model is about 0.45 GB, and I obviously have no issue publishing that model to the service.

I am working on replacing the SharePoint > data flow connection with a direct connection to our Oracle database to eliminate some of the wonky workarounds of exporting CSV files, storing them in SharePoint, etc. However, when I connect to a view of the same fields in the database (again, about 17-18M rows) and load that data in via import mode, the size of the file balloons to 1.1 GB, and that's without even adding in my dimensions! I'm only currently provisioned with a PBI Pro license, so obviously I cannot publish this PBIX file to service.

What am I missing? Does query folding happen when SharePoint/data flows are a source, but not when I connect to Oracle database views? The views were created by another team, and this is the first time we've tried connecting directly to DB for PBI as an organization. Are there best practices I should be sharing with them?

[V-yubandi-msft]

Hi @Dave1mo1 ,

Adding to @GilbertQ  response, the increase in model size when moving from SharePoint CSVs to Oracle views is probably due to differences in data types and compression. Oracle often uses DECIMAL, FLOAT, or wide VARCHAR fields, which typically don't compress as efficiently as the simpler structure of CSVs.

High cardinality columns like timestamps or IDs can also increase memory usage. Additionally, CSVs might have pre filtered data, while Oracle views could include more raw data.

Query folding may also play a role. Although Oracle supports folding, it only works if your Power Query steps are compatible. More complex views with joins or custom logic can break folding and cause full data loads. consider working with your database team to simplify views, use efficient data types, and filter data at the source. If the dataset is still over 1 GB, Power BI Premium Per User might be necessary.

Thanks for you prompt response @GilbertQ .

Warm regards,
Yugandhar.

[GilbertQ]

Hi @Dave1mo1 

I'm going to make an assumption that when you're. From Oracle it is changing the data types and it is these data types of all your model. To increase in size make sure that you use the correct data tab such as whole numbers. Instead of decimal numbers etc. 

What you could also do is to compare your existing semantic model to new your new semantic model using the ALM toolkit and see the differences there.