Forum Discussion
Long Time Waiting
My dataset is taking a long time to update (I'm on the desktop). Too long, like two hours. I'm importing 10 million rows from an OLTP database (it's not ideal, but it's the only alternative for now)....
- Create partitions and load simultaneously. If you can't create partitions like in SSAS, then you can create several tables in Power Query with exactly the same spec apart from a filtering condition. I don't know what fields you have in the big table, but you could certainly retrieve a portion of rows based on some condition(s). Just retrieve different portions into different tables. If you have a date field, then you could create partitions based on year or month...
However, if the problem is with your data source not being capable of retrieving rows quickly enough... You're out of luck. There are system performance counters that are able to tell you what the true speed of retrieval is. Use it to diagnose the problem.
Best
D
That's generally a long time for 10 million rows although it depends on a number of factors. How compex is your query, any merges, things like that? Any unnecessary things like reordering columns? Perhaps simplify it to just get the raw data in with as little transformation as possible? Have you experimented with the query performance tools?
My query uses only a few JOINS and a WHERE filter. I think this is not complex ... What performance tools do you refer to? I'm new to Power BI and I don't know about these tools.