How are teams using AI agents to automate modern data warehouse workflows?
I am exploring how AI agents can help improve data warehouse operations by automating repetitive tasks and assisting data teams with faster decision-making. Some areas I am interested in: Automating data pipeline monitoring and issue detection AI-assisted SQL generation and optimization Automatically identifying data quality issues Triggering workflows based on business events or data changes Generating documentation and insights from warehouse metadata With platforms like Microsoft Fabric Data Warehouse, how are teams approaching AI integration? Are you using: Fabric pipelines with AI-powered automation? Copilot or LLM-based assistants for warehouse development? Custom AI agents connected with data warehouse APIs? Would love to hear real-world architecture patterns and best practices from data engineers working with Fabric.An Overview of Lakehouses and Data Warehouses
Unlocking the Future of Data: Lakehouses vs. Data Warehouses In today’s data-driven world, choosing the right architecture is crucial for turning information into insight. Are traditional data warehouses still the gold standard, or are modern lakehouses rewriting the rules? Dive into our latest article as we explore how these two powerful approaches stack up — and discover which one could transform your data strategy.Microsoft Fabric Data Warehouse
Microsoft Fabric Data Warehouse (Azure Synapse) Microsoft Fabric introduces a new era in data management, and at its core lies a transformative capability—the Data Warehouse, also known as Azure Synapse within Fabric. While it builds on familiar concepts from traditional BI systems, Fabric’s Data Warehouse breaks new ground by offering a truly open, scalable, and fully integrated analytical environment. This article offers a high-level overview of Microsoft Fabric Data Warehouse, providing foundational insights for professionals ready to embrace modern data architecture.
Mirrored Sql Server Database Failed: Internal system error occurred.
When I try use the fabric mirroring to copy data from a on-premises sql server database, I followed the tutorial to create an on-premises data gateway and enable the cdc of database, and make sure the connection is ok. But it turns out the error below, I couldn't find any details from that page or monitor (cause there was no mirror activity in the monitor), also the troubleshoot from Microsoft Fabric documents didn't help.
MySQL Connection Issue via Gateway in Fabric Copy Job
Hi everyone, I'm facing a connection issue when running a Copy Job in Microsoft Fabric using a MySQL source via On-premises Data Gateway. Error: "The integration runtime is busy now. Please retry the operation later." Activity ID: c433b05a-b0a2-434f-9c27-f7b2d858da50 Context: Source: MySQL (on-prem) Connection: Data Gateway Issue occurs during connection/test Checked: Gateway is online Credentials are correct MySQL is reachable from the gateway machine I can successfully connect to the same MySQL source using the same gateway from Power BI Has anyone experienced this issue or knows how to resolve it? Thanks in advance!
OneLake security roles via REST API not syncing to SQL analytics endpoint
We’re using a script to apply data access roles via the Fabric REST API (PUT /dataAccessRoles). The API returns HTTP 200, and the roles plus members show up correctly via the List API, so the roles are applied on the lakehouse. But the SQL analytics endpoint is not picking up these changes. The ols_ roles do get created in the SQL endpoint, but the members are not synced. As a result, sys.database_role_members stays empty. The sync only seems to trigger when changes are made through the OneLake Security UI in the Fabric portal. Evidence Our script ran successfully, HTTP 200, all roles with members confirmed via List API SQL endpoint shows ols_ roles created, but zero members in sys.database_role_members After manually removing and re-adding a member via the UI for a role, all members appeared immediately Impact New users or groups added via deploy do not get SQL endpoint access until a manual sync is triggered via the UI Current workaround Per role in the OneLake Security UI: remove a member, save, add it back, save. This triggers the sync. Can this be investigated and resolved?
DW Missing features compared to SQL server
Hi, Currently so happy with SQL server. Its a high performing db system. its just flys. Im trying to move into lakehouse assuming it will be performing better and noticed many rich sql features are missing some basic ones too. 1 - A simple identity column is missing. i know identity was recently added. However its only big int? bigint takes more space than other types. was it designed in a way so more compute is used? we use tinyint for dim.status and int for keys as needed. 2 - im not able to see dependency of a table used by sp or view or notebook (even if it is parameterized within notebook) i can query the dependencies as well in sql server. 3 - i know underlying table storage is actually delta file. however it would be good if such alter column data types are allowed. 4 - sometimes concurrent updates on a table throws error. 5 - is there any where i could find the list of features that are in SQL but not in DW? Just trying to find where DW is a much better system than Azure SQL server db. yet to uncover more.
Azure SQL mirroring to Fabric fails with Error occured while interacting with landing zone
Hi, I have a strange issue when mirroring an Azure SQL database to Fabric. -When doing GUI point-and-click, the mirroring starts within minutes. Status = Running, Rows Replicated = Correct number -When doing (what I think) is same with API, using same connection, creating mirroring and start, everything looks ok, Status = Running, Rows Replicated = 0. When running SELECT * FROM sys.dm_change_feed_errors; "Error occured while interacting with landing zone" additional: -Im using ServicePrincipal for the connection, using same connection object in both cases. -Using Fabic notebook to initiate the API calls, authenticated as myself (admin). Payload { "properties": { "source": { "type": "AzureSqlDatabase", "typeProperties": { "connection": xxxxxxxxxxxxxx, "database": xxxxxxxxxxxxxxxx } }, "target": { "type": "MountedRelationalDatabase", "typeProperties": { "defaultSchema": "dbo", "format": "Delta" } } } } The issue is reproducable. Whenever point-and-click it works. whenever API it fails.Azure SQL Firewall
Hi there, I have 2 environments - Dev and Prod (Both have same issue). I'm more of admin on Azure environment (recently made as subscription admin) after which I'm having difficulty to remove IP from Azure SQL Firewall. (Those IPs are my own IPs added via SSMS before my i became Azure Subscription admin) (Earlier i was able to delete IPs) today my manager granted me subscription admin and as SQL Security Manager and it still not able to remove grayed out IPs. Prod Resource group has lock. Looks like permission are messed up. What permission do I need to have control over RGroups? Thanks
