Forum Discussion

HamidBee's avatar
HamidBee
Icon for Power Participant rankPower Participant
2 years ago
Solved

Using Dataflow Gen2 to query CSV files in a Lakehouse

Hello All,   I have a folder in a Lakehouse. In that folder I have populated it with CSV files. Now I would like to use Dataflow Gen 2 to build a dataflow that can query all of these CSV files, tra...
data engineering
data warehouse
  • HamidBee's avatar
    HamidBee
    2 years ago

    Thanks for your reply. I got it working in the end. After I closed chrome and relaunched it it showed the csv files went to get data. I think it was an error on my end. Just a note for anyone who is wondering. Inorder to transform multiple csv files at once (assuming they have the same layoiut) I had to create a dataflow, query the files one by one using the get data (within the same dataflow) and then apply the transformations accordingly then nest it inside a pipeline.

HamidBee's avatar
HamidBee
Icon for Power Participant rankPower Participant
2 years ago

Hi. I tried this method but it does not work. Here is what my lakehouse shows me:

 

When I use the dataflow method her is what it shows me:

 

 

It does not display the csv files. Please help.

 

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Did you try to click on the arrow to the left of Files? 

 

Will that expand the Files folder?

  • HamidBee's avatar
    HamidBee
    Icon for Power Participant rankPower Participant
    2 years ago

    Yes I tried this and nothing loads.

    • v-cboorla-msft's avatar
      v-cboorla-msft
      Icon for Microsoft Employee rankMicrosoft Employee
      2 years ago

      Hi HamidBee 

       

      Apologies for the issue that you are facing here.

      I tried to repro the same scenario from our end and didn't find any issue in viewing csv files in Get data.

      Lakehouse that contains csv files.

      Able to see the csv files in Get data.

       

      Try to click the arrow on the left side of the Files and check whether you are able to see the files. If not could you please send me the screenshot of the Get data after clicking the arrow? 

      If issue still persists please do let us know.

       

      I hope this information helps.

       

      • HamidBee's avatar
        HamidBee
        Icon for Power Participant rankPower Participant
        2 years ago

        Thanks for your reply. I got it working in the end. After I closed chrome and relaunched it it showed the csv files went to get data. I think it was an error on my end. Just a note for anyone who is wondering. Inorder to transform multiple csv files at once (assuming they have the same layoiut) I had to create a dataflow, query the files one by one using the get data (within the same dataflow) and then apply the transformations accordingly then nest it inside a pipeline.