Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Shortcut table not showing in SQL Endpoint nor Dataset

Hello.  I am able to successfully establish a shortcut from one lakehouse to another.  I am the owner of both and both exist in the same workspace (also owner there).     I can see the data in my l...
admin
data engineering
security
charrington's avatar
charrington
Icon for Advocate V rankAdvocate V
2 years ago

Hi guys,
any update in this? I have the same thing here.


It seems to me it's either a limitation or bug at this point. I have this scenario:
-LH1 has shortcut to ADLS Gen2 table, which is showing in SQL endpoint, fine.

-LH2 has shortcut to OneLake LH1 same table (virtual with shortcut). That doesn't show in SQL endpoint.

-LH2 has shortcut to OneLake LH1 to some other physical table. That shows in the SQL endpoint. I tested also OneLake shortcuts from other physical tables from other warehouses, they all show in SQL endpoint.

 

So in my case it seems only when adding a shortcut from a Lakehouse, which is itself a shortcut, this doesn't show in SQL endpoint. But still the wizard makes it available, so it should work in theory, meaning to me it's more a bug than anything else...

I added both in the LH2 to double check. 1st one here is shortcut from ADLS, and 2nd is OneLake shortcut referencing the same table (which is clearly parquet as you can see):

But in the SQL view I see only the 1st one, althought they reference the same exact table on ADLS Gen 2 !!

Why??

v-cboorla-msft I think in your case it's working because you do shortcut to some physical table inthe Lakehouse. If you try to shortcut to some existing shortcut, you should not be able to see them on the SQL endpoint.
Please confirm.

thanks

  • FelixL's avatar
    FelixL
    Icon for Advocate II rankAdvocate II
    2 years ago

    I reported this to MS support, who verified the issues and patched it during the November 11 release. After this, it worked fine (again) for me for a few weeks. However, the isse returned last week, and shortcuts to external tables in other lakehouses are now missing in the SQL endpoint again. 

     

    I have reported this to MS again (yesterday) and they are now investigating. I will update you here once I hear more. 

     

    However, it appears to me that MS does not really know themselves if this should or should not work. I get conflicting communication back from support, sometimes referring to the fact that shourtcuts to external AGLS Gen 2 tables are not supported (at all), and sometimes saying this is a bug and that they will investigate. 

     

    OneLake shortcuts - Microsoft Fabric | Microsoft Learn

     

     

    My view on this is that it would be very unfotrunate if this is not supported/will not be supported. That would mean that using external tables (in my case DeltaLake tables created in Azure Synapse Analytics) would relegate me to a second grade fabric citizen, not able to use the most basic functionality of shortcuts throughout the business. 

     

    Another issue related to this has recently been introduced; if a user tries to create the OneLake shortcut to an external lakehouse table (ADLS Gen 2 DeltaLake table) they must have the underlying cloud data connector shared with them, else they receive an error.

     

    This also goes against the whole idea of using the OneLake security model for sharing data - since we really dont want to share the underlying data connectors (containing credentials and root access to data lake) with all business users, since they could then in practice bypass the entire security model by creating their own external shortcuts to whatever data they have access to through the cloud connectors. 

     

    This was not the case until Fabric went GA - before GA all users could create shortcuts to tables in all lakeohuses shared with them, regardless if they had access to the underlying data connector. This more correlated with how I see users working with Fabric's security model; sharing of data sohuld be done on Lakehouse / object level - not also on underlying data connectors.

     

    This has also been reported as a bug to MS support. 

     

    This all goes to say that MS does not seem to really know what will work and what wont work with external shortcuts. And as it now stands the functionalit is at the very least bugged/broken... 

    • charrington's avatar
      charrington
      Icon for Advocate V rankAdvocate V
      2 years ago

      Thanks for the additional info!
      The limitation you highlighted makes no sense to me, that article talks in detail on how ADLS shortcuts work, so how can they be not supported?? Looks more like a documentation not up to date from Microsoft. 

      Yeah some of the Fabric stuff is still in the early stage, even though it's GA. I'm sure they'll improve it over time, we need to be patient.

      • 12angrymentiger's avatar
        12angrymentiger
        Icon for Advocate III rankAdvocate III
        2 years ago

        That's what I'm thinking. It doesn't make any sense to not support ADLS Gen 2 Delta Shortcuts. That's the whole point of the OneLake shortcuts. 

         

        I was informed by a Microsoft Support Engineer that the deployment for the fix went out last Friday. Anonymous  Mentioned there is a fix in place. Hopefully it either gets rolled out further or there is another issue that they are working on.