Forum Discussion

Atul0215's avatar
Atul0215
Frequent Visitor
1 year ago
Solved

How to Fetch 11M+ Rows from OData Entity in Power BI with Incremental Refresh?

I have a task where I need to fetch data from an OData entity with over 11 million rows and multiple columns into Power BI (either through Power BI Desktop or Fabric). However, I do not have SQL acce...
General Comment
  • Atul0215's avatar
    Atul0215
    1 year ago

    Hi Anonymous ,

    Since our database is very large, we plan to initially load all historical data up to the last 6 months. Going forward, we will enable incremental refresh to automatically load only new and updated data.

Atul0215's avatar
Atul0215
Frequent Visitor
1 year ago

Hello Anonymous 

Thank you for your response. I have already tried the solutions mentioned by Akash_Varuna , but I am still unable to refresh or load the data, even in Dataflows. I only need to fetch a single entity that contains a very large amount of data.

Regarding incremental refresh, my understanding is that it only retrieves new or changed data, whereas I need the complete historical dataset, which is quite large.

I am now exploring the BYOD (Bring Your Own Database) solution to handle this large dataset. Would this be a good approach in my case?

Any further guidance would be greatly appreciated.

Akash_Varuna's avatar
Akash_Varuna
Super User
1 year ago

Hi Atul0215 Bring Your Own Database is a good approach but please do look care of the cost and hosting of the database and according to the budget 
If this post helped please do give a kudos and accept this as a solution
Thanks In Advance