connect to data on Hortonworks using webHDFS

[glenncogar]

Hi, we have Hortonworks HDP installed on premises.

We have successfully connected to data on data on HDP via Hive i.e. load data into HDFS using Ambari, create Hive external table, access Hive table from Power BI using Hive ODBC driver.

We are now trying to connect to data on HDP more directly (without Hive) using Power BI and accessing the file e.g. a .csv file stored on HDP.

I am trying to connect using WebHDFS REST services on HDP.  We are doing this via Knox (for security). 

In Power BI I have tried using a Web connection and issuing the REST call in the form of a URL e.g. GETHOMEDIRECTORY or LISTSTATUS of a directory and these work but I can't get the OPEN (read a file) call to work.  It returns an authentication error.

The basic call looks like this...

https://[knox server name.]:[knox port]/[path to REST services]/[file I am trying to open]?op=OPEN

Knox is using basic authentication and so Power BI prompts me for user and password... after that it appears to try and then comes back with an authentication error.

I have also tried to use Power BI's Hadoop connector but can't get this to work either (it just asks for the server in the first instance and fails at that point if I give the Knox server).

If I issue the same REST calls via URL's in my browser it returns the data I am expecting i.e. the content of the file on HDP.

Anyone out there that has made this combo work?

- Power BI

- connect to Hortonworks HDP using WebHDFS REST calls

- can OPEN a data set stored on HDP

Any help appreciated 🙂

[v-jiascu-msft]

Hi @glenncogar,

1. I would suggest you call the REST API directly to check if the Knox causes this issue. Just for test.

2. Did you try the Hadoop connector? Please refer to https://www.youtube.com/watch?v=v-meEu9BRkA.

Best Regards,

Dale

[glenncogar]

Thanks Dale 🙂

I will watch the video on how to use the Hadoop connector, my first experience was that it was not intuitive, the server they ask for could be one of a few types as there are different ways to connect to Hadoop and ask for data, we just happen to use Knox

Re connecting directly to the REST API, I think I did that via a browser and putting in the URL to connect to the REST API via Knox but if that's not what you mean, is there a way of connecting via a non-Knox means? I guess the URL would need to be different, at least the server and port parts and I dont know what they would need to be

Can you elaborate on how to connect directly?