Forum Discussion

Prabunathan's avatar
Prabunathan
Frequent Visitor
8 years ago
Solved

Append daily Excel data into existing Query

Hi,   I have a dashboard created on PBI Pro using dataquery derived from an excel master file.    Now i am looking for an automated process, where in my daily sales data file  (excel) could be ap...
Need Help
  • Anonymous's avatar
    Anonymous
    8 years ago

    Prabunathan,

    If you discard the files in the folder, I am afraid that you will get errors in the report when you refresh the dataset in power BI Service.

    If you concern about the amount of the files, consider to add new data in your original Excel file.

    Regards,
    Lydia

Anonymous's avatar
Anonymous
Not applicable
8 years ago

Prabunathan,

In your scenario, you can put all the files in a single folder, then use the Folder connector in Power BI Desktop to connect to the folder and load data. 

After you create report in Power BI Desktop and publish the report in Power BI Service, set schedule refresh for your dataset. This way, when you add new files to the folder, all the new data will be appened into the existing query.

Regards,
Lydia

  • Prabunathan's avatar
    Prabunathan
    Frequent Visitor
    8 years ago

    Hi Lydia,

     

    Thank you for your suggestion and yes that works fine. 

     

    Kindly advise if the data refresh would consume more time due to the fact that there will be a build up several data files in the folder ( say after a period of 60 days there would be 60 Plus files to collate the data from each time I refresh the data ) . 

     

    Is there a possibility that, i append the latest file to the data query and discard the file each day. Can this be automated placing the data file in a one drive folder.

     

    Thanks again inadvance.

     

    Prabu

     

     

    • Anonymous's avatar
      Anonymous
      Not applicable
      8 years ago

      Prabunathan,

      If you discard the files in the folder, I am afraid that you will get errors in the report when you refresh the dataset in power BI Service.

      If you concern about the amount of the files, consider to add new data in your original Excel file.

      Regards,
      Lydia

      • StacyP's avatar
        StacyP
        New Member
        6 years ago

        Thanks for this reply.  I too am concerned about the performance of the buildup of append files over time for this same scenario.  Related to the option of exporting the data and adding it to the master file - I want to do this, but I'm having issues extracting all of the transformed data from edit queries, as it will only allow me to export 1000 rows from my transformed data query to Excel.  

         

        As well, I have also tried adding a table visual, adding all the columns I want, and exporting the transformed data from the front end.  This also has a significant limitation and is not allowing me to export all rows. 


        Any suggestions?  I need to grow my dataset using append similar to the original poster, but am running into roadblocks with extracting all of the transformed data.