Forum Discussion

sse's avatar
sse
Regular Visitor
10 years ago
Solved

Spark HDinsight slow / 10 mill rows

 

Hi. I am trying to load a quite big data set from Spark HDinsight (beta). In my experience its very slow (approx. 1 hour per mill rows). Any other experiences out there? I am using the Power BI desktop.

 

 Untitled picture.png

Need Help
Show and Tell
  • ALeef's avatar
    ALeef
    10 years ago

    You could optimize the query if you only need to bring back certain columns or date ranges (rows generally).  Most of the time, I only need a few columns, or a certain date range, and that speeds up queries significantly.

1 Reply

  • ALeef's avatar
    ALeef
    Icon for Resolver II rankResolver II
    10 years ago

    You could optimize the query if you only need to bring back certain columns or date ranges (rows generally).  Most of the time, I only need a few columns, or a certain date range, and that speeds up queries significantly.