Mirroring SQL Endpoint Capacity Usage
Hi All, I've been working extensively with the Open Mirroring Database item in Microsoft Fabric, but I'm still trying to fully understand how its capacity (CU) consumption is calculated. From my understanding, the internal operations include components such as: OneLake Write via Redirect OneLake Iterative Read via Proxy OneLake Read via Redirect OneLake Read via Proxy OneLake Write via Proxy OneLake Other Operations My assumption is that these workloads account for activities such as data ingestion, CSV-to-Parquet conversion, data maintenance, vacuum operations, and other background processing required to keep the mirrored database operational. What I do not understand is the Warehouse SQL Endpoint Query Usage metric. On 11 August, this metric increased dramatically in our environment and has remained significantly elevated ever since. For one particular Open Mirroring Database item, usage increased from approximately 7K CU to 34K CU, and I am trying to determine what contributes to this workload and why it changed so suddenly. Before: Screenshot After: Screenshot Another observation is related to the Run By filter in the Capacity Metrics App. Prior to 10 August, I was able to filter consumption by individual user identities and clearly distinguish activity generated by specific users from activity generated by the system. However, from around 10/11 August, all user-generated activity appears to be grouped under a generic "User" identity, and individual users can no longer be distinguished or filtered. Before 10 August: Screenshot showing identifiable users After 10 August: Screenshot showing only "User" I don't understand what this new User identity represents. Is it an aggregation of all user-generated activity, a new classification introduced by the metering changes, or something else entirely? My current theory is that both the increase in Warehouse SQL Endpoint Query Usage and the change in the Run By filter may be related to the recent Fabric capacity metering changes, but I have not been able to find any documentation that clearly explains this behavior. Can anyone provide insight into the following? What specifically contributes to Warehouse SQL Endpoint Query Usage for an Open Mirroring Database? Do any of the internal mirroring processes execute through the SQL Endpoint and therefore appear under this metric? Did the recent Fabric metering changes alter how these operations are classified or billed? What exactly does the "User" identity represent in the Run By filter? Why did individual user identities disappear after 10 August and become a generic "User" value? Is the new User Identity dimension connected to the recent metering changes? Has the attribution model for user-generated workloads changed, resulting in individual users being grouped into a single category? Any clarification would be greatly appreciated, as these changes make it difficult to understand capacity consumption patterns and compare usage before and after 10 August. Thanks!
Mirror replication Premium extremely high consumption issue
Dear Community, We have switched on the CDF opportunity on 1-2 Mirrored database. We have noticed 30% minutes later we are almost knocked out the F32 license fully. We have on it extremnely small data. 10-20K records. Even the size is small also. Just rising to the sky without no reason. On the dev we have 20 table with 3K records....
Oracle Mirroring - Invalid Identifier Error
I am trying to mirror from an on-prem Oracle DB, and one of the tables I need gives me the following error: SourceErrorCode: ReplicationError: Failed to do initial data load for table <TableName> due to 'ORA-00904: "SYS_NC00112_": invalid identifier', ErrorCode: InputValidationError" I know this is an invalid column header error, and I tracked down the issue to a hidden system generated column named, "SYS_NC00112$". They're similar, but the column name in the error message has the dollar sign replaced with an underscore. Is this a special character limitation with the dollar sign? Is there any way around this?Fabric SQL Server mirroring - fails on time-out
I have encountered issues when mirroring large tables (2.5 billion records and more) from SQL server 2022 using the Fabric mirroring through on-prem data gateway. The issue seems to be a time-out which is enforced after 4 hours of the initial load running. On Fabric side I can only see "Internal system error occured" but on the data gateway side I can see it is actually time-out which seem to line-up to a 4 hour window: Has anyone experienced this before? Is there a solution? I haven't found anyhing on the gateway side of things to help.
The database cannot be mirrored to Fabric due Data Factory disabled
I was trying to setup On prem SQL mirroring in a development Workspace but was given error messages below: New SQL Mirror: The database cannot be mirrored to Fabric due to below error: Test connection failed. Data Factory 801d6d4c-9d6f-4126-924d-8cae3d820a7e is in state Disabled. The requested operation is not permitted. Activity ID: 3a8b4029-9bed-45cf-9339-503a36b70625 Choose Data: Data Factory 801d6d4c-9d6f-4126-924d-8cae3d820a7e is in state Disabled. The requested operation is not permitted. Activity ID: b7188a47-b26f-43d9-9bda-28f581c2c26f Refresh or troubleshoot the issue. I went digging around with fab-cli command, but dont think that shows Data Factory <uuid> for me to debug further. Any suggestions to solve this issue is appreciated. I managed to create the SQL mirror in my other workspace. Another strange behavior I found in this workspace is when i tried to create an item "Copy Data". I will get thrown with different error. Create item "Copy Job" {"message":"Request failed with 400(Bad Request): {\"error\":{\"code\":\"FactoryOperationNotPermitted\",\"pbi.error\":{\"code\":\"FactoryOperationNotPermitted\",\"exceptionCulprit\":1}}}, \nrequestid: 41e8edb5-b0cb-8c2b-4c7b-93f7d2ee24eb"} Thanks & Regards, Richard
Some of many tables in my mirror show a "running with warnings" message in Fabric Mirror Monitor.
We have setup mirroring to an AZ SQL database and that went just fine. Tables were initialised and replicated succesfullly. However, for the last few days I have noticed some tables are showing the "running with warnings" message in the mirror monitor and the last completed date is in the past and not consistant with the data in the source table. It appears changes are no longer being reflected into the mirror for those tables with warnings. The message behind the bubble in monitor is not so reveling and simply states 'Internal system error occured. Artifactid = .....' Our source AZ DB for the mirrored tables is configured with the Business Critical (BC_Gen5_12) SKU and does have a resonably heavy workload. Can anyone help to shed some light on my issue. Anonymous v-cboorla-msft
Open Mirroring Issue
Hello Everyone , I am trying the Open Mirroring feature which is in preview. Whenever I am trying this I am getting some error, see below screenshot where for the error. the steps I did is first I uploaded the _metadata.json file and then I uploaded the parquet file but still getting the below error. Did anyone tried it and its working for you ?
