Forum Discussion

LaurentZ's avatar
LaurentZ
Helper I
6 years ago
Solved

Get Data from Folder wich are not 100% the same

Hi,   I've 4 CSV (with TAB as separator) files each month that I need to append. So I use the "Import from Folder" feature in PowerQuery.   Those CSV files are extracted from SAP in a way that I...
  • mahoneypat's avatar
    6 years ago

    FYI that PowerQuery in Excel also use M language; the query editor is the same in both.  What you are describing is doable.  Here is one way.

     

    1.  Use the Folder connector to go to the folder with your 4 csv files, and choose Combine & Transform (or something close to that)

    2. That will generate a bunch of stuff in the query editor, including a function and a Transform Sample File query (that is tied to that function)

    3.  In the Transform Sample File query, do the following transformations

    - remove top 11 rows

    - promote headers

    -filter out blanks (in any one of the columns)

    - Ctrl-click the 3 (or 4) columns you want to keep, right click and choose Remove Other Columns

    4.  Go to your main query at the bottom (the one with an Invoked Custom Function step that used the function, now updated with your new steps).  If all the remaining columns are the same in the 4 files, all the data should be combined in that query.

    5.  Do any additional transformations needed

     

    If this works for you, please mark it as the solution.  Kudos are appreciated too.  Please let me know if not.

    Regards,

    Pat