Forum Discussion

MatjoCol's avatar
MatjoCol
Frequent Visitor
5 years ago

Reading parquet from ADLS Gen 1

Hi!

It is easy to read parquet files/folder using the parquet connector 
It is also easy to read csv etc from data lake gen 1 (adress example adl://aplha.azuredatalakestore.net/beta/gamma)

But if I try to read a parquet file, 
1) I cannot connect to the data lake since the connector only accept http/https
2) If I try to use the data lake connector then the files show up as binary files.

You can probably combine/parse these files but that would be quite complicated.
Is it possible to read parquet stored in azure data lake as easily as if they were stored locally? 

Need Help

2 Replies

    • MatjoCol's avatar
      MatjoCol
      Frequent Visitor
      5 years ago

      Hi Anonymous!

      ADLS Gen2 connector only works with a https schema so I cannot connect the gen 1 adress I have adl://. And even if I had a Gen2 data lake where I could put the data I do not think you can have the connector pointing towards a folder. Maybe the parquet connector can point towards a Gen 2 endpoint, I will examine that. 

      The reason for me having to point to a folder is that I do not control the names and how many partitions the parquet files that I want to connect to. 

      Best regards,
      Mattias