Forum Discussion

cpshort7186's avatar
cpshort7186
Advocate I
1 year ago
Solved

Poor ODBC insert performance in Warehouse

I'm trying to insert records into a Warehouse table with Alteryx and still extreme performance issues.  Data output is just a few thousand rows.  Job runs in 9 sec without trying to write to Fabric W...
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi cpshort7186 
    Thank you for reaching out microsoft fabric community forum.

    • While there isn’t a direct "enable bulk loading" checkbox in Alteryx, you can achieve bulk loading by using the MS SQL Server Bulk connection type with the ODBC Driver 18 for SQL Server. This method bypasses row-by-row inserts and uses bulk operations, which can significantly improve performance.

    • Fabric Warehouse has been reported to perform better with smaller batch sizes. Try with committing data in batches of 1,000 rows (or fewer) to reduce transactional overhead.

      Instead of writing directly to Fabric Warehouse, try the following:
      •  Write your data to a Lakehouse in Parquet format (this is highly performant for ingestion). Create a Shortcut in the Fabric Warehouse to query the data from the Lakehouse.
      • This approach leverages the strengths of Lakehouse for ingestion and Warehouse for querying.

    If Alteryx continues to underperform for your use case, consider evaluating other ETL tools that may offer better integration with Fabric Warehouse. Tools like Azure Data Factory or Synapse Pipelines are designed to work seamlessly with Microsoft’s data platforms.

    If the issue persists, we recommend raising a support ticket for further assistance. To create a support ticket for Fabric and Power BI, please refer to the steps outlined in the following guide:

    How to create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn



    If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community.
    Thank you.