Forum Discussion

sundayDriver's avatar
sundayDriver
Microsoft Employee
9 years ago
Solved

Import logs files(txt format) from network share

Hi,

I wanted to import some log files for text processing. The files exist in multiple locations inside a network share.

The main flow is: Go to the root network share and then search all the subfolders to get all the files of type .txt.

Is there a way that I can do that?

Also does PowerBI support any processing of text, a.k.a could I use it of processing logs, extracting common error messages etc?

 

Thank you,

Lydia

Need Help
  • Anonymous's avatar
    Anonymous
    9 years ago

    Hi sundayDriver,

    Do all the text files have same structure(contain same number and name of columns)  ? If that is the case, you can perform the following steps to connect to them in Power BI Desktop. However, if the text files don't have same structure, Power BI Desktop will not merge the results from files correctly.

    1. Open Power BI Desktop, and choose “Get Data->Blank query”, then click Advanced Editor, paste the following code in it. Please replace network share path with your own.


    let
    Source = Folder.Contents("\\VYUEZH074204VM\test"),
    TypeAdded=Table.AddColumn(Source,"Type",each Value.Is([Content],type table)),
    Folders=Table.SelectRows(TypeAdded, each [Type]=true)
in
    Folders

    2. Choose Content, Extensions and Folder path to expand the Content column as shown in the following screenshot.


    3. Filter Content.Extension column to “.txt”.


    4. Expand the Content.Content column by clicking double-arrow, then click OK in the pop-up windows.

     

    Reference:
    http://radacad.com/fetch-files-andor-folders-with-filtering-and-masking-power-query

    Thanks,
    Lydia Zhang

2 Replies

  • hugoberry's avatar
    hugoberry
    Responsive Resident
    9 years ago

    Firstly use Get Data> Folder and get all the files in that root folder. Then filter the resultset to the files that you need. 

    Afterwards you can expand the column with the contents of those log files, hoping that Power BI will merge them correctly.

  • Anonymous's avatar
    Anonymous
    Not applicable
    9 years ago

    Hi sundayDriver,

    Do all the text files have same structure(contain same number and name of columns)  ? If that is the case, you can perform the following steps to connect to them in Power BI Desktop. However, if the text files don't have same structure, Power BI Desktop will not merge the results from files correctly.

    1. Open Power BI Desktop, and choose “Get Data->Blank query”, then click Advanced Editor, paste the following code in it. Please replace network share path with your own.


    let
    Source = Folder.Contents("\\VYUEZH074204VM\test"),
    TypeAdded=Table.AddColumn(Source,"Type",each Value.Is([Content],type table)),
    Folders=Table.SelectRows(TypeAdded, each [Type]=true)
in
    Folders

    2. Choose Content, Extensions and Folder path to expand the Content column as shown in the following screenshot.


    3. Filter Content.Extension column to “.txt”.


    4. Expand the Content.Content column by clicking double-arrow, then click OK in the pop-up windows.

     

    Reference:
    http://radacad.com/fetch-files-andor-folders-with-filtering-and-masking-power-query

    Thanks,
    Lydia Zhang