VNet Gateway with Azure Databricks SQL Warehouse

Hi Anonymous ,

VNet gateways allow import or direct query semantic models to connect to data services within an Azure virtual network.

You can try to connect in Direct Query mode.

Best Regards,
Dengliang Li

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

direct query has latency issues, import is required for the dashboard scope.

Hi Anonymous,
Thank you for reaching out to the Microsoft fabric community forum. Thank you Anonymous, for his inputs on this issue.

After reviewing the details you provided, I have identified few workarounds that may help resolve the issue. Please follow these steps:

• Optimize Data Imports: Limit the data being imported by applying filters and consider using Incremental Load to import only new or updated data, which can reduce the load and minimize conflicts.
• Data Model Optimization: Ensure that your data model is efficient by minimizing complex joins and aggregations. Partitioning large datasets can help distribute the load and reduce locking issues.
• Concurrency Control: Implement query throttling or batch processing to avoid simultaneous operations, and schedule imports during low-traffic periods to reduce conflicts.
• VNet Gateway Tuning: Review your VNet Gateway configuration to ensure it’s optimized for performance and capacity.
• Scaling Resources: Scale your Databricks cluster for better resource allocation, ensuring smooth processing during imports.

Additionally, you can explore the following Microsoft documentation for more guidance on optimizing VNet Gateway configurations and best practices for working with Databricks:

Virtual Network Data Gateway Best Practices

Troubleshoot the Virtual Network Data Gateway

Use Virtual Network Data Gateway in Power BI

If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using Microsoft Community Forum.