Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
6 years ago

Data refresh is taking too long

Hi all,

 

The refreshing of my data is taking a lot a time. One of the cause is that I connect to a SQL server.
That table could have 600 000 rows let's say. But I copied/paste that query to do some other transformations to create new table.

Meaning that for each refresh it will load twice (or more depending on how many new tables I need that uses this data) 600 000 rows. Is there a way not to?

 

For example if I create an empty table and I append that new table to the one with 600 000 rows, will the data be loaded only once? (It does not feel like it).

 

Hope it makes sense.

 

Kr,

 

David

 

 

 

 

Need Help

5 Replies

  • Greg_Deckler's avatar
    Greg_Deckler
    Community Champion
    6 years ago

    Try referencing your original query (right-click, Reference) instead of copy paste.

     

  • Anonymous's avatar
    Anonymous
    Not applicable
    6 years ago

    Hi Anonymous,

    Any advanced operation in your query steps? (merge, combine, reference other query or query steps, duplicate query tables or custom functions... ) If this is a case, you can consider using buffer functions or other methods to increase calculation performance.

    How and When to use List & Table Buffer? 
    Regards,

    Xiaoxin Sheng

     

  • Anonymous's avatar
    Anonymous
    Not applicable
    6 years ago

    Hello,

     

    Sorry for the late reply.

     

    The best solution I would like to achieve is to implement incremental refresh.

    But I am not able to make it work.

     

    After a lot of research I don't know what I'm doing wrong.

    Conditions for incremental refresh:

    -Power BI premium

    -Create datetime parameter Start/End

    -Filter the date column with those parameters

    -Make sure query can be folded

    -Set up incremental refresh

    -Publish

    -Do first history data refresh

     

    Is there anything I forgot?

    In my case I don't think it is working because the first refresh takes +-10 and the second as well (and there is no new data).

    I set up the history to 24months and the refresh in the last 1 days. The column it is based on do not have datetime within last 1 day so it should not refresh everything right?

     

    Kr,

     

    David