POST a CSV to API

[LDTi247]

I am using Fabric to process some data and am trying to get it sent to an internal API. 

I have set up multiple connections and VNETs, attempted Copy data activities and user data functions within Data Pipelines, but I'm experiencing issues. 

The copy activity doesn't want to pick up the file to send to the API, and I get an error message "no file sent", indicating that I am connecting to the API, but the file isn't. 

With the User Data Function, I have got the function working in VS Code, but when it is published to Fabric and run through the Pipeline, it has problems connecting to the API endpoint. 

What else am I missing and how can it get it to work? 

[v-kpoloju-msft]

Hi @LDTi247,

Thank you for the detailed explanation that really helps. From what you have described, there are two things happening.

Copy Data activity: The error “no file sent” means the API is not receiving the file in the right format. Copy Activity does not support posting files as multipart/form-data, which is what most APIs expect for CSV uploads.

User Data Function: Since it works in VS Code but not in Fabric, the issue is connectivity. Fabric runs in a managed environment, so to reach your internal API you will need to set up Managed VNET integration and a Private Endpoint.

Confirm the payload format the API expects (multipart/form-data, raw file, or Base64 in JSON). Use a Notebook in Fabric with Python requests to send the file in the correct format, then call that Notebook from your pipeline. If you stick with User Data Functions, configure Managed VNET + Private Endpoint and test connectivity with a simple GET request from a Notebook.

Hope this helps. If you have any doubts regarding this, please feel free to ask here. We will be happy to help.
Thank you for using the Microsoft Community Forum.

[LDTi247]

Hi @v-kpoloju-msft, 

I believe that when I test the UDF in VS Code, it works as it does on the local network, allowing access to the endpoint. 

All of the documentation that I have managed to find about the private endpoints suggests that it will only work for SQL endpoints over the VNET. Please let me know if i have missed something. 

My current solution is to put all of the file locations in a table in a lakehouse, use this as a database for a GraphQL API. Then using Airflow I can get the file location, download the file from the onleake API and post it to my internal end point. This all seems a little excessive and I would hapily take a simpler method. 

Thanks. 

[v-kpoloju-msft]

Hi @LDTi247,
Thanks a lot for sharing those details and for clarifying your current approach that helps us understand where you are coming from.

You are right that most of the Fabric docs/examples around Private Endpoints focus on SQL endpoints. However, Managed VNET + Private Endpoints can be used for any supported Azure PaaS resource or custom endpoint, not just SQL. The key is that the endpoint must be reachable over a Private Link. If your internal API is hosted on an Azure service that supports Private Link (e.g., App Service, Storage, Key Vault, etc.), you should be able to expose it securely to Fabric through a Private Endpoint.

Check Private Link availability: Confirm whether the internal API hosting platform supports Private Link. https://learn.microsoft.com/en-in/azure/private-link/private-link-overview

Test connectivity from Fabric: You can spin up a Notebook in Fabric and issue a simple requests.get() call to verify if the Private Endpoint is working.

Alternative option: If your API cannot be exposed via Private Link, then your workaround with Airflow does make sense, though it is more complex. In that case, a middle-tier service like Azure Functions or API Management (which supports VNET integration) could help simplify the flow, acting as a bridge between Fabric and your internal API.

You haven’t missed anything, but the docs you found are SQL-heavy because that’s a common case. Private Endpoints are broader than that and might still help you here.

Hope this clears things up. If you have any doubts regarding this, please feel free to ask here. We will be happy to help.

Thank you for using the Microsoft Community Forum.

[v-kpoloju-msft]

Hi @LDTi247,

Just checking in to see if the issue has been resolved on your end. If the earlier suggestions helped, that’s great to hear! And if you’re still facing challenges, feel free to share more details happy to assist further.

Thank you.