Forum Discussion

cdok1091's avatar
cdok1091
Advocate II
9 years ago

dynamically import files from a folder WITHOUT combining them

Hi all, I'm looking to upload multiple files of the same type from a single folder into Power BI Desktop.  However, in this case, I don't want to combine them into a single query.  How can I dynamic...
Need Help
cdok1091's avatar
cdok1091
Advocate II
9 years ago

Disclaimer: I'm new to both Power BI and R (only a couple months into learning both). 

 

I wrote an R script that dynamically generates dataframes for each file of a specified file type from a specified folder and then saves those dataframes to an R environment '.RData' file.  Then, in Power BI Desktop, I use R Script as a datasource and use 'load("filename.RData") to load the dataframes from that environment without running the entire underlying script that generated those dataframes -- which gives me control over 'refresh' (manually accomplished in R) and saves processing time in Power BI.  When an R script is used as a data source, it pulls compatible R objects (dataframes, matrices, lists) into Power BI as separate queries/data sources.  This is what I wanted to achieve.  However, there seems to be a limit to the number of R objects Power BI can load into a Desktop session.   I can't load all ~120 dataframes, so now I'm using trial and error to find my limit for successful dataframe loading.  Regardless, there does seem to be a limit, but I'm not sure if the limit is due to an internal Power BI constraint (number of objects and/or the size of the objects), or due to my own system constraints (RAM, etc).

  • ImkeF's avatar
    ImkeF
    Community Champion
    8 years ago

    Hi ajuanto,

    this is the error-message I'm getting when trying to open the link you've posted:

    So please post link that can be accessed publicly, thanks.

    • ajuanto's avatar
      ajuanto
      Frequent Visitor
      8 years ago

      Hi @Imke,

      Here is the message > \

       

       

      Hi I am new to Power BI and I am facing an issue.

       

      I have a Folder of csv files which I have to import. The csv files has readings of a electronic device,

       

      I imported the csv folder and I got such a screen

       

      Then I tried to remove the unwanted columns

       

      now I have a column that shows the sourse file that is in my case the csv file for eg-MFT_RB1_2017-08-08_11-26-56.csv

      I am getting all the data i.e the combined data of over 100 csv file in one sigle table as shown above. I want to create diffrent tables for each csv files. I can do it manually by importing csv files seperately like this -

       But when I am importing a folder I am getting a sigle combined table, I want to split the main table into tables=number of csv files and visualiye the output as above

      (I got the above visualization by importing 6 csv one by one, which works perfect)

      But when importing a complete folder I am getting one big table with all 100 csv combined and I want to create diffrent tables for diffrent csv like I did with the above visualization.

      Also, I have over 100 csv files in the folder and if I add 50 more in the folder it should automatically create tables of those newly added csv files.

       

      Can anyone help me with this?

       

      • ajuanto's avatar
        ajuanto
        Frequent Visitor
        8 years ago

        Is this possible or is there any other way that I could visualize the values as I need it?