Forum Discussion
Helper IWhat is the practical difference between referencing data via shortcut in ADLS gen2 vs. OneLake?
Hi ,
Thanks for explaining your ask.Yes, there are some features that the Fabric team would be losing by having the data in ADLS Gen2 and reading it via shortcuts vs. having it in OneLake.
Performance: OneLake tables are optimized for performance, especially for queries that involve joins and aggregations. The Fabric team may experience slower performance for queries that involve joins and aggregations if they are reading the data from ADLS Gen2 via shortcuts.
Security: OneLake tables can be secured with role-based access control (RBAC), which makes it easier to manage who has access to the data. The Fabric team would need to manage permissions to the ADLS Gen2 storage account and the Delta Parquet tables if they are reading the data from ADLS Gen2 via shortcuts. This could be more complex than managing permissions to OneLake tables, especially if there are multiple teams accessing the data.
Governance: OneLake tables can be governed with policies, which makes it easier to ensure that the data is used in a compliant manner. The Fabric team would need to develop and implement their own policies to govern the use of the data in ADLS Gen2 if they are reading the data from ADLS Gen2 via shortcuts. This could be more complex than using the built-in policies in OneLake.
In addition to these features, OneLake also offers a number of other advantages, such as:
- Data sharing:OneLake makes it easy to share data with other teams and applications, both within and outside of your organization.
- Data governance:OneLake provides a number of built-in data governance features, such as RBAC, auditing, and data lineage.
- Data quality:OneLake provides a number of data quality features, such as data profiling and data cleansing.
Overall, OneLake is a more comprehensive and robust data management platform than ADLS Gen2. If you are planning to use Fabric to access the data, it is recommended to use OneLake to store and manage the data.
If the Fabric team needs the best performance, security, and governance features, then the data can be stored in OneLake. However, if the Fabric team is only concerned with reading the data, and does not need the additional features of OneLake, then it is possible to read the data from ADLS Gen2 via shortcuts.
Hope this helps. Please let me know if you have any further queries.
I have attached some links. Please refer them for more information:
https://www.infoworld.com/article/3704608/understanding-onelake-and-lakehouses-in-microsoft-fabric.html
https://radacad.com/saying-yes-to-fabric-onelake-shortcuts-no-to-duplicate#:~:text=For%20example%2C%20a%20Shortcut%20can,ADLS%20Gen2%20or%20Amazon%20S3.
I believe you are correct, however I have not seen any documentation that clarifies if Direct Lake Mode in Power BI is possible when leveraging v-ordered Delta Parquet tables via shortcut to ADLS Gen2, or if the v-ordered Delta Parquet tables must truly live in OneLake.
Closest thing I have found is the below graphic seems to indicate that as long as you have a a Fabric/Premium capacity, a Lakehouse, and v-ordered Delta Parquet tables, it should work.
Hi mwjones61 ,
Thanks for using Fabric Community.
As I understand you are trying to compare Onelake and Shortcuts here. But Shortcuts are objects in Onelake that point to data in other storage locations. You can create shortcuts in lakehouses and Kusto Query Language (KQL) databases.
There are many advantages of Shortcuts in Onelake:
Shortcuts in Fabric offer a number of advantages:
- Data sharing:Shortcuts allow you to share data between different users and applications without having to copy the data. This can help to improve data consistency and reduce data duplication.
- Data virtualization:Shortcuts can be used to create virtual data sets that combine data from different sources. This can make it easier to access and analyze data from different sources.
- Data governance:Shortcuts can be used to implement data governance policies. For example, you can use shortcuts to restrict access to data to certain users or applications.
- Performance: Shortcuts can help to improve performance by reducing the need to move data between different systems.
- Cost savings:Shortcuts can help to reduce data storage and processing costs by reducing the need to copy data between different systems.
Could you please be more clear on the ask as I am not able to understand which two things you are trying to compare?Please do let us know if you have any further queries.
You can refer to the documentation : link