Forum Discussion

PBILover's avatar
PBILover
Icon for Helper V rankHelper V
6 years ago

Incremental data refresh for large dataset

Hi , trying to implement a incremental dataset refresh on the large dataset  (around 1 million row) on a premium workspace ,but at the first time that it is initial load (loading historical data) ge...
Need Help
PBILover's avatar
PBILover
Icon for Helper V rankHelper V
6 years ago

thanks for your reply.

Its almost 69 columns in  which some are custom columns.

I tried to load only half million rows but still it is giving timeout error.

 

 it gives Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

 

 

 

lbendlin's avatar
lbendlin
Icon for Super User rankSuper User
6 years ago

that is usually a transient condition (meaning you may get different results when you try again). what's the data source?

  • PBILover's avatar
    PBILover
    Icon for Helper V rankHelper V
    6 years ago

    the data source is a transactional data for communications and source is from cosmos .

    did i answer your question correctly? or do u need more info on what's the data source?

    • lbendlin's avatar
      lbendlin
      Icon for Super User rankSuper User
      6 years ago

      I don't think the Cosmos driver is exposing the QueryTimeout settings. Try pulling in a narrower dataset (fewer columns) .

      • PBILover's avatar
        PBILover
        Icon for Helper V rankHelper V
        6 years ago

        ok, I am trying to do that now.

        But this dataset was working fine in a production environ,ent before and suddeny satrted giving diff kinds of error.

         

        Is there any other way  how to divide large dataset into a smaller one and combine the data of all in one for the reporting purpose?