Forum Discussion

J-Dixon's avatar
J-Dixon
Frequent Visitor
1 year ago
Solved

Materialized Lake Views - Couldn't load lineage - name already exists in the DAG

Trying out the new Materialized Lake Views in a lakehouse on Fabric

 

I've created two views in two different schemas with the same name. The use case is each schema represents pre-filtered populations of records, where the attributes and structure of the table are the same

 

When attempting to load the lineage view of the MLVs, I get the error in the title.  I assume the tool cannot handle multiple MLV with the same name despite different schema 

 

 

  • After some sessions with Microsoft support, this was recognised as a bug and fed back to the product team 

     

    In the meantime, there is a workaround 

     

    In the MLV definitions, if you use only lowercase when referencing a schema, the feature will work correctly 

     

    This includes situations where the schema in the lakehouse uses capital letters. The definitions seem to be case-agnostic

     

    In my scenario, I have schemas with capital letters. I created several MLV with interdependencies ensuring all references were in lowercase, and the output data and MLV lineage view is all correct

14 Replies

  • J-Dixon's avatar
    J-Dixon
    Frequent Visitor

    Update - 

     

    I changed the name of the second MLV and the same error still occurs, so this isn't due to same naming 

     

    I dropped all MLVs and started from scratch. Adding first MLV works as expected, adding the second (different name) which depends on the first, causes the error described 

  • Joniswe's avatar
    Joniswe
    Regular Visitor

    HI J-Dixon

     

    I have same problem,  I have not found root cause yet.

    I have different names on my mlv:s but if I remove all mlv:s except the first one it works

     

    BR Jonas

     

    • v-kpoloju-msft's avatar
      v-kpoloju-msft
      Icon for Community Support rankCommunity Support

      Hi J-Dixon,
      Thank you for reaching out to the community and sharing the details of your scenario.

      Based on the error "name already exists in the DAG" and your scenario, it appears that the lineage view in Microsoft Fabric currently does not support multiple Materialized Lake Views (MLVs) with the same name, even if they are in different schemas.

      Recommended Action: Please try renaming one of the MLVs so that each has a unique name, even if their structure is identical. Once this is done, retry the lineage view it should load without issues. We recognize that using identical names across schemas is a valid architectural pattern, and this may be a limitation in the current DAG engine.

      Kindly refer to the below mentioned links for better understanding:
      Manage Fabric Materialized Lake Views Lineage - Microsoft Fabric | Microsoft Learn
      Materialized lake views Spark SQL reference - Microsoft Fabric | Microsoft Learn

      Hope this helps clarify things and let me know what you find after giving these steps a try happy to help you investigate this further.

      Thank you for using the Microsoft Fabric Community Forum.

      • J-Dixon's avatar
        J-Dixon
        Frequent Visitor

        Hi, as mentioned in the other comment, removing and re-creating the MLV with a different name causes the same error. I'm seeing the same behaviour as the other commenter, this error happens whenever a second MLV is created, despite different naming 

  • J-Dixon's avatar
    J-Dixon
    Frequent Visitor

    After some sessions with Microsoft support, this was recognised as a bug and fed back to the product team 

     

    In the meantime, there is a workaround 

     

    In the MLV definitions, if you use only lowercase when referencing a schema, the feature will work correctly 

     

    This includes situations where the schema in the lakehouse uses capital letters. The definitions seem to be case-agnostic

     

    In my scenario, I have schemas with capital letters. I created several MLV with interdependencies ensuring all references were in lowercase, and the output data and MLV lineage view is all correct