Forum Discussion
Copy Data Activity with REST API POST - Issue with request body and escape characters
I am trying to use a copy data activity to get data from a REST API using a POST call and copy it to a lakehouse. The problem is that the copy data activity adds escape characters to the double quotes \" which makes the body unreadable by the API. I tried to use variables and concatenating and replacing \" with " but nothing seems to work.
Is there a solution to this issue?
Anonymous I agree that using a Copy activity is a better choice in this case compared to a Web activity. But something just does not add up in this picture: Web activity can execute the call, but the Copy activity fails. Both activities should be using the same REST connector with the same request settings. Can you compare settings of Web and Copy activities and make sure both are configured the same way? You could also paste a schreenshot of your Web activity settings here for a second check.
15 Replies
Anonymous I don't think that COPY activity did anything wrong. The encoding on the wire is correct and is based on how the REST connector handles content when the content-type is application/json. On the other hand, response 406 does not indicate an incorrect request, which would be 400 Bad Request. I guess, something is wrong based on 406 response status, but I would not blindly blame the COPY activity. To prove that, I did a quick experiment with a public REST API using a POST with a JSON body. As you would guess, the experiement was successful. See below the results:
As you can notice the encoding in the activity input is exactly the same as in your case, which is how a well-formed JSON looks like on the wire, and the response is also correct.
However, getting back to 406 response, it usually means that the request is missing an accept header that should indicate a content type that the response should be generated for. It seams that your API is very picky about that header and in your configuration it's missing. Try to add accept: application/json header to your COPY activity if you expect a JSON response, or another proper type, and verify if it works correctly this time.How does your accept header look like?
Anonymous Can you post a screenshot of your COPY activity configuration for the body of your POST request? It sounds from your description that something is missing or misconfigured.
It is correct that the Accept header is not usually needed when the content-type header is specified. In my previous example above I did not need the Accept header. From information you provided so far it's difficult to diagnose the precise problem. You body and header look absolutely correctly from the activity input screenshot. For your original question about the body encoding the answer is yes, the encoding of the body is correct. But you say that the API returns an error. Can you post the output from your activity run or maybe a response that you get when you test with other tools like Postman?
Hi Anonymous, thanks for your reply. I don't think the ouput from Postman is helpful with debugging issue in Fabric. I was asking about the output from your pipeline run in Fabric to see specifically what error was returned by the API.
Hello, yes I just replied to apturlov with the output I see in Postman for the same call
Hi Anonymous ,
May I ask if you have resolved this issue? If not, kindly share the requested details.
Thank you.
