Blog Post

Fabric Updates Blog
3 MIN READ

Using Bulk Copy API for faster ingestion in Fabric Data Warehouse (Preview)

jovanpop-msft's avatar
jovanpop-msft
Icon for Microsoft Employee rankMicrosoft Employee
3 months ago

BCP API provides a client-side ingestion path for cases where data is produced in application code and must be written directly to warehouse tables without staging.

Figure: Ingesting data from the client applications with BCP API provides better throughput and ingestion speed.

The recommended ingestion approach in Fabric Data Warehouse remains server-side loading with COPY INTO whenever you can stage files in storage first. That remains the primary path for managed, file-based warehouse loading. BCP API is for a different class of problems, where client applications need to send data directly without an extra staging step in a lake or storage account.

Use BCP API when direct ingestion is required

Choose BCP API when your source data exists in the client tier and you cannot stage files first. In these scenarios, using BCP API is usually more efficient than executing many row-by-row INSERT statements. Typical direct-ingestion scenarios include:

  • Client-side services that receive event batches or read local files and write them with SqlBulkCopy (C#) or SQLServerBulkCopy (Java).
  • A script-driven operations workflow runs bcp.exe for controlled imports from terminal or runbook jobs.
  • A data integration workflow uses tools such as SSIS, Informatica, Azure Data Factory, or dbatools, where SQL bulk-load connectors can use BCP API under the hood.

How to use it in day-to-day SQL workflows

BCP API is typically used in direct-ingestion scenarios by three audiences: application developers building ingestion services, SQL engineers and DBAs running scripts, and data integration users working in orchestration tools.

Role 1: Application developers building ingestion services

If your team owns C# or Java ingestion services, you can call bulk copy classes directly from application code by using SqlBulkCopy in C# or SQLServerBulkCopy in Java. This approach is a good fit when data is already in memory and must be written directly to a table.

  • C# example: this pattern uses the SqlBulkCopy class to perform high-throughput client-side writes through BCP API.
using var bulk = new SqlBulkCopy(connectionString);
bulk.DestinationTableName = "dbo.Sales";
await bulk.WriteToServerAsync(dataTable);
  • Java example: this pattern uses the SQLServerBulkCopy class to perform the same bulk-ingestion model from Java workloads.
SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(connectionString);
bulkCopy.setDestinationTableName("dbo.Sales");
bulkCopy.writeToServer(resultSet);

In both cases, the client sends batches through bulk copy semantics instead of issuing many single row INSERT statements, which improves ingestion throughput.

Role 2: SQL engineers and DBAs running scripts

If your team runs ingestion through scripts, runbooks, or the command line, bcp.exe is the command-line utility for bulk copy operations. The following command uses the BCP utility to load file data through the same BCP API model.

bcp dbo.Sales in sales.csv -S <workspace-endpoint> -d <database> -G -c -t ,

This is useful for operational jobs where DBAs and SQL engineers need repeatable command-line imports.

Role 3: Data integration users working in orchestration tools

If your team uses integration platforms such as Azure Data Factory, SSIS, or Informatica, you can apply the same client-side bulk-ingestion model through connector and task configuration rather than through application code.

Conclusion

BCP API support provides a direct client-side ingestion path for workloads that cannot stage files before ingestion. It enables practical scenarios across application services, script and runbook operations, and integration tools while reducing row-by-row insert overhead.


With BCP API (Preview), you can evaluate new data ingestion scenarios across these client-side patterns. Try it out in your application services, scripts, or integration pipelines, and compare performance with your current approach.

  • Learn how to use SqlBulkCopy in C# or SQLServerBulkCopy in Java to validate high-throughput client-side ingestion.
  • Learn how to use the bcp.exe utility in script and runbook workflows for repeatable command-line bulk loads.
  • Use your existing SSIS, Azure Data Factory, or Informatica pipelines and tasks that use SQL bulk copy semantics for faster client-side ingestion.

BCP API supports these patterns and provides faster client-side ingestion over SQL connections.


We’d love your feedback as you try it—your input during preview helps shape the experience before general availability. If you encounter any issues or have questions, you can provide feedback on the Data Warehouse - Microsoft Fabric Community site.

Updated 3 months ago
Version 1.0
No CommentsBe the first to comment