Data Ingestion using API

[Rafaela07]

Hi everyone,

I'm working on a new project in Fabric and I'm trying to ingest data from an external API into a Lakehouse using Data Factory Copy Activity.
The client provided the url, and a username and password for basic authentication. They also provided some parameters, eg ?limit=100
For the copy activity I created a rest connection using the url and the authentication credentials and the connectivity test is successful. In the relative url part I've put something like /api/campaigns?limit=50 and I tried to preview the respective data.
However I'm bumping into the following error:
Rest call failed with server error, status code 500 InternalServerError, please check your activity settings and remote server issues.
Request URL: XXXXXXXXXXXXXXXX/api/campaigns?limit=50.
Response: {"errors":[{"message":"Looks like I encountered an error (error #500). If I do it again, please report me to the system administrator!","code":500,"type":null}]}"

Am I missing something? I've never ingested data using apis before so I'm not sure what is going wrong.
Any suggestions regarding this situation?

Thank you in advance,

[v-lgarikapat]

Hi @Rafaela07 

Thank you for reaching out to the Fabric Community Forum.

@deborshi_nag , @tayloramy  Thanks for your prompt response.

I wanted to follow up to check if you had a chance to review the information provided by @deborshi_nag , @tayloramy  If you are still facing issues, please don’t hesitate to reach out  we’ll be more than happy to assist you.

Best Regards,

Lakshmi.

[Rafaela07]

Hi @v-lgarikapat!

I've tried both @tayloramy and @deborshi_nag approaches with the notebook request and via postman yet I'm still encountering the same 500 error. I contacted the API vendor and waiting for their response.

Best,

[v-lgarikapat]

Hi @Rafaela07, Thanks for the update

Thanks,

Lakshmi. 

[deborshi_nag]

Hi @Rafaela07 , a return code 500 is a general server error, and since you're seeing the same result in Postman, it indicates the issue is not on the client or caller side. You've taken the correct step by reaching out to the API provider, and hopefully they will be able to address the problem. Apologies that I couldn't offer more assistance.

[tayloramy]

Hi @Rafaela07, 

If you're encountering the issue in Postman, then the issue has nothing to do with Fabric. Hopefully the vendor can help solve the problem with their API. 

[deborshi_nag]

Hi @Rafaela07 

I would suggest trying out the REST endpoint using a desktop tool like Postman or Insomnia. 

Hope this helps - please appreciate by leaving a Kudos or accepting as a Solution! 

[tayloramy]

Hi @Rafaela07, 

A 500 error means that you successfully connected to the API, but something on the backend of the API went wrong. 

The steps I would take to troubleshoot are: 
1: create a text notebook and see if you can get data from the API using Python. That gives you far more flexibility to know exactly what URL is being hit. 

2: If the notebook encounters the same issue, contact the API vendor 

3: If the notebook works, then there is something different between the URL the pipeline is using and the URL the notebook is using.