Forum Discussion
Poor ODBC insert performance in Warehouse
- Anonymous1 year ago
Hi cpshort7186
Thank you for reaching out microsoft fabric community forum.- While there isn’t a direct "enable bulk loading" checkbox in Alteryx, you can achieve bulk loading by using the MS SQL Server Bulk connection type with the ODBC Driver 18 for SQL Server. This method bypasses row-by-row inserts and uses bulk operations, which can significantly improve performance.
- Fabric Warehouse has been reported to perform better with smaller batch sizes. Try with committing data in batches of 1,000 rows (or fewer) to reduce transactional overhead.
Instead of writing directly to Fabric Warehouse, try the following:- Write your data to a Lakehouse in Parquet format (this is highly performant for ingestion). Create a Shortcut in the Fabric Warehouse to query the data from the Lakehouse.
- This approach leverages the strengths of Lakehouse for ingestion and Warehouse for querying.
If Alteryx continues to underperform for your use case, consider evaluating other ETL tools that may offer better integration with Fabric Warehouse. Tools like Azure Data Factory or Synapse Pipelines are designed to work seamlessly with Microsoft’s data platforms.
If the issue persists, we recommend raising a support ticket for further assistance. To create a support ticket for Fabric and Power BI, please refer to the steps outlined in the following guide:
How to create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn
If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community.
Thank you. - While there isn’t a direct "enable bulk loading" checkbox in Alteryx, you can achieve bulk loading by using the MS SQL Server Bulk connection type with the ODBC Driver 18 for SQL Server. This method bypasses row-by-row inserts and uses bulk operations, which can significantly improve performance.
Hi cpshort7186
Could you please confirm if the issue has been resolved after raising a support case? If a solution has been found, it would be greatly appreciated if you could share your insights with the community. This would be helpful for other members who may encounter similar issues.
Thank you for your understanding and assistance.