Forum Discussion
Pulling Big Data From MYSQL database
Hi,
I have a large data table stored in Mysql database and while pulling the same i 'm getting query timeout error as IT admin has set query timeout for security reason which he can't change.
Is there any alternative way to pull all this data?
Just having this Idea : Is that possible to pull the data in bundles of small periods and keep appending them in a single table and then if somehow i would use incremental refresh on the same.
Hoping some great suggesting from experts...!!!
4 Replies
Hi Namrata_Das_CLR ,
Yes you can pull the data on the smaller batches and append the tables to create the new table in which you can schedule incremental refresh. In this way, you can you can retrieve data in smaller periods, consolidate it into a single table in Power Query, and enable incremental refresh to keep the consolidated table up to date with new data. This helps overcome the query timeout limitation while efficiently managing the data retrieval and refresh process in Power BI.
Best Regards,
Ajith Prasath
If this post helps, then please consider Accept it as the solution and give kudos to help the other members find it more quickly.
Hi @AjithPrasath, i tried your suggestion but the appended table is not supporting incremental refresh
Hello Namrata_Das_CLR ,
Try to create views and get only the data needed so you could have less load.
If you want all the data, try using direct query https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-use-directquery
If I answered your question, please mark my post as solution, Appreciate your Kudos 👍
Thanks Idrissshatila , as you are aware mysql database does not support direct query, is there any alternative way.
