topic Re: Getting data from elasticsearch in Data Engineering

Getting data from elasticsearch

wahmed20 — Wed, 31 May 2023 14:49:27 GMT

What is a good way to get data from elasticsearch into fabric [DataGen2]?

Re: Getting data from elasticsearch

HimanshuS-msft — Wed, 31 May 2023 23:07:58 GMT

Hello @wahmed20 ,
Looking at the data flow gen2 and data facory , I do not see any connetor for Elasticsearch .
But we do have the REST connector and we can use that to pull data using data facory pipeline .

I will also like you t o point to this implementation .

https://stackoverflow.com/questions/66991666/azure-data-factory-rest-api-paging-with-elasticsearch

Thanks
Himanshu

Re: Getting data from elasticsearch

wahmed20 — Thu, 01 Jun 2023 12:15:23 GMT

@HimanshuS-msft In the Setting there is managed gateway

in which there is connection type elastic. I created that

then in my data pipeline , I use copy data

In the source when I test connection it gives error

The error is