cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
alisoroush7
Frequent Visitor

refreshing a huge number of rows in a table

hi guys,

the problem is i have a table that contains 20 million of records. every time i want to refresh the data i should wait about 30 to 45 minutes to complete. the question is that is there any way to update 5 last million records of a table? or any other solution to avoid waiting too much?

Thanks

1 ACCEPTED SOLUTION
v-henryk-mstf
Community Support
Community Support

Hi @alisoroush7 ,

 

There are a few options you can consider to try and reduce the time it takes to refresh your data in Power BI:

  • Use DirectQuery mode: Instead of importing all of the data into Power BI, you can use DirectQuery mode to connect directly to the data source and query the data on demand. This can be faster, especially if you have a fast, efficient data source and a well-designed database. However, it may not always be possible to use DirectQuery mode, depending on the data source and the requirements of your report.
  • Optimize the data source: If you are using an on-premises data source, there are several ways you can try to optimize the data source to improve performance. For example, you can try indexing the tables in the data source, or partitioning the data to spread it across multiple tables or servers.
  • Use incremental refresh: If your data source supports it, you can use incremental refresh to only refresh the data that has changed since the last refresh. This can be a significant time saver, especially if you have a large dataset that changes frequently.
  • Use a faster computer: If you are using a slower computer or one with limited resources (e.g. CPU, RAM), this can contribute to the slow refresh times. Upgrading to a faster computer with more resources may help.
  • Use a faster connection: If you are using a slow internet connection, this can also contribute to slow refresh times. Using a faster connection (e.g. a wired Ethernet connection instead of WiFi) may help.

If the problem is still not resolved, please provide detailed error information and let me know immediately. Looking forward to your reply.


Best Regards,
Henry


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

3 REPLIES 3
v-henryk-mstf
Community Support
Community Support

Hi @alisoroush7 ,

 

There are a few options you can consider to try and reduce the time it takes to refresh your data in Power BI:

  • Use DirectQuery mode: Instead of importing all of the data into Power BI, you can use DirectQuery mode to connect directly to the data source and query the data on demand. This can be faster, especially if you have a fast, efficient data source and a well-designed database. However, it may not always be possible to use DirectQuery mode, depending on the data source and the requirements of your report.
  • Optimize the data source: If you are using an on-premises data source, there are several ways you can try to optimize the data source to improve performance. For example, you can try indexing the tables in the data source, or partitioning the data to spread it across multiple tables or servers.
  • Use incremental refresh: If your data source supports it, you can use incremental refresh to only refresh the data that has changed since the last refresh. This can be a significant time saver, especially if you have a large dataset that changes frequently.
  • Use a faster computer: If you are using a slower computer or one with limited resources (e.g. CPU, RAM), this can contribute to the slow refresh times. Upgrading to a faster computer with more resources may help.
  • Use a faster connection: If you are using a slow internet connection, this can also contribute to slow refresh times. Using a faster connection (e.g. a wired Ethernet connection instead of WiFi) may help.

If the problem is still not resolved, please provide detailed error information and let me know immediately. Looking forward to your reply.


Best Regards,
Henry


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Greg_Deckler
Super User
Super User

@alisoroush7 Yes, you can use Incremental Refresh. https://www.bing.com/ck/a?!&&p=1eb4908b95b30592JmltdHM9MTY3MTIzNTIwMCZpZ3VpZD0zMWJiZmJkOC1lYzJlLTY2N...

 


@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
Mastering Power BI 2nd Edition

DAX is easy, CALCULATE makes DAX hard...

but incremental refresh is for premium users only!

 

Helpful resources

Announcements
PBI Sept Update Carousel

Power BI September 2023 Update

Take a look at the September 2023 Power BI update to learn more.

Learn Live

Learn Live: Event Series

Join Microsoft Reactor and learn from developers.

Dashboard in a day with date

Exclusive opportunity for Women!

Join us for a free, hands-on Microsoft workshop led by women trainers for women where you will learn how to build a Dashboard in a Day!

MPPC 2023 PBI Carousel

Power Platform Conference-Power BI and Fabric Sessions

Join us Oct 1 - 6 in Las Vegas for the Microsoft Power Platform Conference.