Forum Discussion

StanislavPtacek's avatar
StanislavPtacek
New Member
5 months ago
Solved

Fabric SQL Server mirroring - fails on time-out

I have encountered issues when mirroring large tables (2.5 billion records and more) from SQL server 2022 using the Fabric mirroring through on-prem data gateway. The issue seems to be a time-out which is enforced after 4 hours of the initial load running. On Fabric side I can only see "Internal system error occured"

but on the data gateway side I can see it is actually time-out which seem to line-up to a 4 hour window:

Has anyone experienced this before? Is there a solution? I haven't found anyhing on the gateway side of things to help. 

  • Hi StanislavPtacek  ,
    Thanks for reaching out to the Microsoft Fabric Community forum.

    These are some of the reasons why it is not possible to Mirror SQL server, Please check if any of these points are causing the issue you are facing:

    1)  Azure SQL Database cannot be mirrored if the database has: enabled Change Data Capture (CDC), Azure Synapse Link for SQL, or the database is already mirrored in another Fabric workspace.

    2) The maximum number of tables that can be mirrored into Fabric is 500 tables. Any tables above the 500 limit currently cannot be replicated.

    3)  Tables with primary key or a clustered index (when a primary key does not exist) on unsupported types cannot be mirrored - computed columns, user-defined types, geometry, geography, hierarchy ID, SQL variant, timestamp, datetime2(7), datetimeoffset(7), or time(7).


    4)  Source tables that have any of the following features in use cannot be mirrored.

     

    • Temporal history tables and ledger history tables 
    • Always Encrypted
    • In-memory tables
    • Graph 
    • External tables

     


    I hope this information helps. Please do let us know if you have any further queries.
    Thank you

  • Hi StanislavPtacek,

     

    As v-nmadadi-msft  already mentioned, these are limitations.
    However, I have encountered this problem myself and solved it as follows.

     

    I used Open Mirroring to do this. Here are the steps I took.

    1. Created a mirrored database
    2. Exported the data from the SQL database to Parquet format using a script.
    3. Created a script that does this automatically for me later after the initial load.

     

    The advantage of open mirroring is that you can prepare your data so that it becomes mirrored.
    You also have control over the size of the individual files you upload.
    As I mentioned, I have automated this process.

     

    You will find all the necessary information in the Microsoft documentation.

    https://learn.microsoft.com/en-us/fabric/mirroring/open-mirroring

     

    I hope this helps, and if you have any further questions, please don't hesitate to contact us.

     

    Best regards

     

    Feel free to leave kudos or accept it as a solution. This will also help other community members.

     

4 Replies

  • Hi StanislavPtacek  ,
    Thanks for reaching out to the Microsoft Fabric Community forum.

    These are some of the reasons why it is not possible to Mirror SQL server, Please check if any of these points are causing the issue you are facing:

    1)  Azure SQL Database cannot be mirrored if the database has: enabled Change Data Capture (CDC), Azure Synapse Link for SQL, or the database is already mirrored in another Fabric workspace.

    2) The maximum number of tables that can be mirrored into Fabric is 500 tables. Any tables above the 500 limit currently cannot be replicated.

    3)  Tables with primary key or a clustered index (when a primary key does not exist) on unsupported types cannot be mirrored - computed columns, user-defined types, geometry, geography, hierarchy ID, SQL variant, timestamp, datetime2(7), datetimeoffset(7), or time(7).


    4)  Source tables that have any of the following features in use cannot be mirrored.

     

    • Temporal history tables and ledger history tables 
    • Always Encrypted
    • In-memory tables
    • Graph 
    • External tables

     


    I hope this information helps. Please do let us know if you have any further queries.
    Thank you

  • Hi StanislavPtacek,

     

    As v-nmadadi-msft  already mentioned, these are limitations.
    However, I have encountered this problem myself and solved it as follows.

     

    I used Open Mirroring to do this. Here are the steps I took.

    1. Created a mirrored database
    2. Exported the data from the SQL database to Parquet format using a script.
    3. Created a script that does this automatically for me later after the initial load.

     

    The advantage of open mirroring is that you can prepare your data so that it becomes mirrored.
    You also have control over the size of the individual files you upload.
    As I mentioned, I have automated this process.

     

    You will find all the necessary information in the Microsoft documentation.

    https://learn.microsoft.com/en-us/fabric/mirroring/open-mirroring

     

    I hope this helps, and if you have any further questions, please don't hesitate to contact us.

     

    Best regards

     

    Feel free to leave kudos or accept it as a solution. This will also help other community members.

     

  • Hi StanislavPtacek 

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.


    Thank you.