Forum Discussion

TK12345's avatar
TK12345
Icon for Resolver II rankResolver II
1 day ago

OAP for MLVs cross workspace

Hi all,

I have a question regarding OAP implementation across multiple workspaces.

My current setup consists of three layers:

Bronze: OAP enabled

Silver: OAP enabled

Gold: OAP not enabled

In the Bronze layer, I extract all source data using specific endpoints and gateways, with the required connection rules configured. In the Silver layer, I use shortcuts to access the source lakehouses in Bronze. This works as expected.

Within Silver, I then create Materialized Lake Views (MLVs) for my entities. For example, I create a Person entity based on multiple source tables and store the result in LH_Silver. This also works correctly.

The issue occurs in the next step. On top of the Silver entity, I want to create dim_Person. The MLV reads the entity from LH_Silver, but the output should be written to LH_Gold in the Gold workspace.

When OAP is disabled on all workspaces, this cross-workspace setup works and I can successfully write the MLV to Gold. However, as soon as OAP is enabled on the Silver workspace, I can no longer write the MLV to the Gold workspace.

I've already tried several connection rules, including:

FabricMaterializedLakeView

FabricSqlEndpointMetadata

Fabric SQL Analytics Endpoint

Lakehouse

Notebook

Web V2

I've also configured an MPE for the Gold (3000) workspace, but so far none of these options have resolved the issue.

Does anyone know which connection rule, MPE, or configuration is required to allow an MLV running from an OAP-enabled Silver workspace to write to a Lakehouse in another workspace?

Ideally, I would like to keep the notebooks and MLV logic in the Silver layer rather than moving them to the Gold workspace.

Thanks in advance!

Cel 1:

%%pyspark

spark.conf.set("spark.sql.caseSensitive", "true")

workspace = spark.conf.get("trident.workspace.name")

environment = "DEV" if "DEV" in workspace else "TEST" if "TEST" in workspace else "PRD"

spark.sql(f"SET TargetEnvironment = 3000_CURATED ({environment})")

 

Cel 2:

CREATE OR REPLACE MATERIALIZED LAKE VIEW `${TargetEnvironment}`.LH_3000.Algemeen.dim_Persoon AS

SELECT *

FROM test.`02_entity`.Person

2 Replies

  • v-saisrao-msft's avatar
    v-saisrao-msft
    Icon for Community Support rankCommunity Support

    Hi TK12345​,

    Based on the documented OAP behavior, please configure a Managed Private Endpoint (MPE) from the OAP-enabled Silver workspace to the Gold workspace and ensure that the corresponding Private Link Service request is approved. This is the documented approach for enabling cross-workspace access from an outbound-protected workspace. Once the MPE connection is established, please retry the MLV materialization. Please note that the current Microsoft documentation does not specifically document the Silver MLV → Gold MLV materialization scenario

    Create an allow list using managed private endpoints - Microsoft Fabric | Microsoft Learn

    About Cross-Workspace Communication - Microsoft Fabric | Microsoft Learn

    Thank you.

    • TK12345's avatar
      TK12345
      Icon for Resolver II rankResolver II

      Hi v-saisrao-msft​

      Thanks for getting back to me.

      Your suggestion to configure an MPE from the Silver workspace to the Gold workspace is indeed something I've already tried, as mentioned in my original post:

      "I've also configured an MPE for the Gold (3000) workspace, but so far none of these options have resolved the issue."

      Unfortunately, this did not resolve the issue either.

      Do you have any other suggestions on how to make this cross-workspace setup work when OAP is enabled on the Silver workspace?

      Thanks in advance!