Using Dataflow Gen2 to query CSV files in a Lakehouse

[HamidBee]

Hello All,

I have a folder in a Lakehouse. In that folder I have populated it with CSV files. Now I would like to use Dataflow Gen 2 to build a dataflow that can query all of these CSV files, transform it and then save it to a new table in the same Lakehouse. The issue I am facing is that I cannot find the correct source from the "Get Data". I can't seem to locate the CSV files.

Any help would be greatly appreciated.

[HimanshuS-msft]

Hello @HamidBee 
Thanks for using the Fabric community.

When you navigate to the getdata and you type Lakehouse you should be able to see that option .

The below screenshot should help .

Thanks
Himanshu




Thanks
HImanshu

[HamidBee]

Hi. I tried this method but it does not work. Here is what my lakehouse shows me:

When I use the dataflow method her is what it shows me:

It does not display the csv files. Please help.

[frithjof]

Did you try to click on the arrow to the left of Files? 

Will that expand the Files folder?

[HamidBee]

Yes I tried this and nothing loads.

[v-cboorla-msft]

Hi @HamidBee 

Apologies for the issue that you are facing here.

I tried to repro the same scenario from our end and didn't find any issue in viewing csv files in Get data.

Lakehouse that contains csv files.

Able to see the csv files in Get data.

Try to click the arrow on the left side of the Files and check whether you are able to see the files. If not could you please send me the screenshot of the Get data after clicking the arrow? 

If issue still persists please do let us know.

I hope this information helps.

[HamidBee · Accepted solution]

Thanks for your reply. I got it working in the end. After I closed chrome and relaunched it it showed the csv files went to get data. I think it was an error on my end. Just a note for anyone who is wondering. Inorder to transform multiple csv files at once (assuming they have the same layoiut) I had to create a dataflow, query the files one by one using the get data (within the same dataflow) and then apply the transformations accordingly then nest it inside a pipeline.

[v-cboorla-msft]

Hi @HamidBee 

Glad that your query got resolved.

Please continue using Fabric Community for any help regarding your queries.