Forum Discussion

SLI's avatar
SLI
Helper IV
7 years ago

Incremental refresh

Hello    can you please tell me, how i can do to get an incremental refresh with power bi (pro licences) if it is not available, how can i do to reduce the time of refresh (i have a table wich con...
Need Help
jeffshieldsdev's avatar
jeffshieldsdev
Solution Sage
7 years ago

My work-around for incremental refresh is to download historical data into flat-files (CSV, JSON, TXT) and put in OneDrive/SharePoint.  I pull these files into Power BI in the Query Editor and Append with query that retrieves "live" data.

 

I query the last date in my "historical" dataset to determine the start date for my "live" query--so it's always dynamic as I drop more "historical" files in my OneDrive/SharePoint folder.

  • steph_io's avatar
    steph_io
    Advocate II
    7 years ago

    Is it possible to write back to your OneDrive/Sharepoint historic data from Power Query?

     

    My scenario is similar. I have an historic static file on sharepoint as my starting point.

    I use Power Query to detemine the incremental records. This is dynamic in a sense that it will compare data of newly added files to this static file. 

    But how do I now write back these incemental records so they now become part of the static, and not longer re-evaluated each refresh?? 

    Is there a way to automate this, and ie: not manually rename the query for each refresh... 

    • jeffshieldsdev's avatar
      jeffshieldsdev
      Solution Sage
      7 years ago

      You don't write back, you just connect to it.

       

      I would start with one or a few of your historical files in a local folder, and use the Folder connector to connect to it.  Power BI will write some Power Query M for you then.  Use Combine & Edit to loop through these files and ingest them into Power BI.

       

      Separately, use the OneDrive connector to connect to one of your files in SharePoint.  Then swap out the "Source =" code in the Folder version with the "Source =" and related lines from the SharePoint version. :)