Forum Discussion

AnnOminous's avatar
AnnOminous
Microsoft Employee
1 year ago
Solved

Can't write data to Fabric DB - MashupException.Error

I'm trying to write data from a Dataflow (not Gen2) to a SQL DB in Fabric. The query is set to refresh incrementally, but when I try to publish/refresh the query, I get the following error message: ...
data engineering
  • BhaveshPatel's avatar
    BhaveshPatel
    1 year ago

    Hi AnnOminous 

     

    You need to use Notebooks only ( Advanced User ). Also You need to understand Data Lake and Delta Lake with Apache Spark. To answer the question,This is writing data back to Fabric SQL ( INSERT ). This is a script.

    # Create DataFrame to insert into the Azure/Fabric SQL  table ( Writing Data to Azure SQL / Fabric SQL Table )
     
    df1 = spark.createDataFrame([('WL2456', 'Luke1293', 'Skywalker','Null','Null',-3,'Unknown','Unknown','Null')], ['DeviceID', 'DeviceName', 'DeviceDesc','OperatorName','OperatorDesc','FacilityID','FacilityName','FacilityDesc','DeviceSerialNo'])

    # Insert the rows into the Azure SQL / Fabric SQL table
    df1.write \
        .option('user', user) \
       .option('password', pswd) \
       .jdbc('jdbc:sqlserver://' + sqlserver + ':' + port + ';database=' + database, 'dbo.Devices', mode = 'append' )
       
       
    df1.show()



    Cheers
    Bhavesh Patel



     

Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi AnnOminous ,

Thank you for reaching out to the Microsoft Fabric Community Forum.

 

To clarify, writing data from a Dataflow (non-Gen2) directly into a Fabric SQL DB (or any Fabric-native destination such as Lakehouse or Warehouse) is not supported. This limitation is what’s leading to the MashupException.Error during the publishing and refresh operations, particularly when incremental refresh and parameterized function logic are involved.

 

As correctly said by BhaveshPatel @, to perform this kind of operation successfully, you’ll need to migrate your solution to Dataflow Gen2, which is fully integrated with the Fabric ecosystem. Dataflow Gen2 supports writing to Fabric SQL DB, Lakehouse, and Warehouse while aligning with the unified data model and schema expectations required by incremental refresh scenarios

Further recommended next steps are as follow:

  • Rebuild the dataflow using Dataflow Gen2.
  • Reapply the incremental logic and destination settings within the Gen2 experience.
  • Confirm your Fabric SQL DB is accessible and correctly mapped in the new Gen2 workspace.

Hope this helps. Please reach out for further assistance.

 

Thank you.