Materialized Lake View - 'Invalid Date, Invalid Date'

[TheFifthJoin]

Recently started seeing some very strange behaviour following updating my OPDG to the August release where if we add or remove a column from a materialized lake view, other columns become 'Invalid Date, Invalid Date'. For example we dropped 'age' from a people table, and the unchanged 'title' column turned to this:

The column is clearly a string and the SQL sat behind this runs as expected (column populated Mr/Mrs/etc).

We rolled back to the previous OPDG in a hope it was something to do with that as its been working fine and the update is the only thing that happened around the time we noticied the issue - but rolling back hasn't helped.

We update the MLV when needed by dropping it, and then creating a new one, so it shouldn't be holding any ghost data types or anything weird. We've even dropped a MLV, gone on lunch, and ran the create when we got back thinking maybe it took a bit to propogate the drop.. but at this point it feels like we're clutching.. it also didnt work and we got the same issue.

It does this on any MLV, from any data source.

recreation:

CREATE MATERIALIZED LAKE VIEW IF NOT EXISTS `silver-ws`.silver_lh.core.people AS...

(SQL..)

->

DROP MATERIALIZED LAKE VIEW IF EXISTS `silver-ws`.silver_lh.core.people;

->

CREATE MATERIALIZED LAKE VIEW IF NOT EXISTS `silver-ws`.silver_lh.core.people AS...

(change the SQL giving a new column or removing an existing one)

->

fields are messed up.

Anyone come across this before, or have any insight as to what the reason might be?

[V-yubandi-msft · Accepted solution]

Hi @TheFifthJoin ,

Thank you for providing the details. Even reverting to a previous version may not fully resolve it, since cached metadata can persist across sessions and environments. @anilgavhane , suggestions are still relevant and worth trying.
For the

"Private link check failed"

This is probably not directly related to the MLV issue, but may be impacted by the same OPDG update. This error often occurs if

1. The Fabric workspace or capacity isn’t properly registered in the Private DNS zone

2. The Private Link endpoint is misconfigured or not fully propagated

3. You’re using On Premises Data Gateway, which doesn’t fully support Private Link

If you’re in a secure VNet environment, you might consider using the VNet Data Gateway, as it’s designed for Private Link and can help avoid these errors.

Alternatively, you can

1. Disable redirection temporarily and test creating the MLV with a new name and explicit casting
2. Check Private DNS zone records to ensure Fabric endpoints are properly registered
3. If using OPDG with Private Link, try switching to VNet Data Gateway

Helpful Reference:
Solved: Seeking Advice: Private Link and VNet Data Gateway... - Microsoft Fabric Community

Thanks for prompt response @anilgavhane .

Regards,
Yugandhar.

[V-yubandi-msft]

Hi @TheFifthJoin ,
Could you let us know if your issue has been resolved or if you are still experiencing difficulties? Your feedback is valuable to the community and can help others facing similar problems.

[TheFifthJoin · Accepted solution]

Hi @V-yubandi-msft 

To update - bit of a mixed bag of solutions.. The rollback of the OPDG seemed to fix the columns going into strange orders when we added/removed a column from the materialized lake view. I've not seen this as a known issue, but rolling back seemed to resolve and we'll wait for a new update release before trying to update the OPDG again.

The 'Invalid Date, Invalid Date' seems to be being displayed when my date value in the raw data is NULL. If we load this via the sql endpoint in SSMS, the fields display as null values as expected.. a bit of an odd display, but I believe its just a display quirk, rather than an actual issue with the data. Hopefully this will get addressed in a future release also, but for now, ignoring it is doing no harm as the 'Invalid Date, Invalid Date' message doesnt pull into powerBI visualisation either.

Finally the Private link issue was a much wider issue that affected all of south-west connections. Something to do with an error in an update MS pushed - they address, rolled back internally, and the error disappeared.

Thanks for checking back!

[V-yubandi-msft]

Hi @TheFifthJoin ,
I wanted to follow up to see if your issue has been resolved or if you still need any assistance. If you need more information, feel free to let me know.

Thank you.

[V-yubandi-msft]

Hi @TheFifthJoin ,

Has your issue been resolved, or are you still running into trouble? Your feedback helps others who might be facing the same challenge.

Thank You

[V-yubandi-msft · Accepted solution]

Hi @TheFifthJoin ,

Thank you for providing the details. Even reverting to a previous version may not fully resolve it, since cached metadata can persist across sessions and environments. @anilgavhane , suggestions are still relevant and worth trying.
For the

"Private link check failed"

This is probably not directly related to the MLV issue, but may be impacted by the same OPDG update. This error often occurs if

1. The Fabric workspace or capacity isn’t properly registered in the Private DNS zone

2. The Private Link endpoint is misconfigured or not fully propagated

3. You’re using On Premises Data Gateway, which doesn’t fully support Private Link

If you’re in a secure VNet environment, you might consider using the VNet Data Gateway, as it’s designed for Private Link and can help avoid these errors.

Alternatively, you can

1. Disable redirection temporarily and test creating the MLV with a new name and explicit casting
2. Check Private DNS zone records to ensure Fabric endpoints are properly registered
3. If using OPDG with Private Link, try switching to VNet Data Gateway

Helpful Reference:
Solved: Seeking Advice: Private Link and VNet Data Gateway... - Microsoft Fabric Community

Thanks for prompt response @anilgavhane .

Regards,
Yugandhar.

[anilgavhane]

@TheFifthJoin

This issue with Materialized Lake Views (MLVs) showing 'Invalid Date, Invalid Date' after schema changes—especially when dropping or adding columns—is likely tied to a metadata caching or schema inference bug introduced in the August OPDG update. You're not alone; similar instability has been reported across Fabric components following recent updates.

What Might Be Happening

• Schema inference glitch: When you drop or add a column, Fabric may incorrectly re-infer data types for existing columns, especially if the metadata cache isn’t fully cleared.
• Ghost metadata: Even after dropping and recreating the MLV, cached schema definitions may persist, causing type mismatches (e.g., treating a string column as a date).
• Propagation delay: The delay you mentioned between drop and recreate might not be enough to flush internal metadata references.

  Suggested Workarounds

  • Explicit Casting in SQL
    Force the correct data type in your MLV SQL:

    SELECT CAST(title AS STRING) AS title, ...

    • Use a New MLV Name Temporarily
      Create the MLV under a different name to bypass cached metadata:

      CREATE MATERIALIZED LAKE VIEW IF NOT EXISTS `silver-ws`.silver_lh.core.people_v2 AS ...

      1. Clear Metadata via Workspace Restart
         If possible, restart the workspace or session to flush cached schema definitions.
      2. Raise a Ticket with Microsoft Support
         This behavior may be tied to the August OPDG release. Others have resolved similar issues by downgrading or applying a patch.

[TheFifthJoin]

We rolled back the OPDG to June's release (our last known good configuration) and that doesnt seem to have resolved the issue at all. 

As of this morning we're now also getting an error when trying to connect a session in a notebook

"reason": "{\"reason\":\"Private link check failed.\",\"state\":\"private-link-check-failed\"}",

Are these likely to be related?