Forum Discussion
Direct Lake Model Resource Limit
Hi Everyone,
I am working on implementing Fabric for our future unified data platform and recently i was building a new semantic model from Fabric Data Warehouse with 2 Dims and 1 fact table. The fact table has over 1.3 Billion records and i was using Direct lake mode however i got a resource limitation error when trying to refresh it in PBI Desktop while creating relation ship. Also our F64 capacity maxed out when i build a sample report on this data with a date slicer. Any suggestions on how to mitigate this?
Since direct lake doesnt replicate data into semantic model it should have acted more like a direct query without taking too much of compute. Has anyone exprienced this?
Hi Ira_27
Direct Lake mode loads data into memory from Delta tables stored in OneLake as needed for query execution. While it doesn’t create a full copy of your data (unlike Import mode), it still needs to page data into memory to process queries. Your F64 capacity has a maximum memory limit of 25GB, which can be quickly consumed when working with a billion row data
exceeding the memory limit in Direct Lake doesn’t automatically trigger a fallback to DirectQuery mode. Instead, it causes excessive paging in and out of model data from OneLake, which significantly impacts performance and can lead to resource errors
When creating relationships in Power BI Desktop, the system needs to analyze the columns involved to determine cardinality and cross-filter direction
Date slicers can trigger complex queries that retrieve all date values and their relationships to fact data, potentially loading large portions of your fact table into memory
This may be the reason F64 capacity maxed out when using a date slicer on your sample report.
6 Replies
Hi Ira_27
Direct Lake mode loads data into memory from Delta tables stored in OneLake as needed for query execution. While it doesn’t create a full copy of your data (unlike Import mode), it still needs to page data into memory to process queries. Your F64 capacity has a maximum memory limit of 25GB, which can be quickly consumed when working with a billion row data
exceeding the memory limit in Direct Lake doesn’t automatically trigger a fallback to DirectQuery mode. Instead, it causes excessive paging in and out of model data from OneLake, which significantly impacts performance and can lead to resource errors
When creating relationships in Power BI Desktop, the system needs to analyze the columns involved to determine cardinality and cross-filter direction
Date slicers can trigger complex queries that retrieve all date values and their relationships to fact data, potentially loading large portions of your fact table into memory
This may be the reason F64 capacity maxed out when using a date slicer on your sample report.
Hi Ira_27,
Thanks for reaching out to the Microsoft fabric community forum.
As nilendraFabric already responded to your query, please go through his response and check if it solves your issue.
I would also take a moment to thank nilendraFabric, for actively participating in the community forum and for the solutions you’ve been sharing in the community forum. Your contributions make a real difference.
If I misunderstand your needs or you still have problems on it, please feel free to let us know.
Best Regards,
Hammad.
Community Support TeamIf this post helps then please mark it as a solution, so that other members find it more quickly.
Thank you.
Hi Ira_27,
As we haven’t heard back from you, so just following up to our previous message. I'd like to confirm if you've successfully resolved this issue or if you need further help.
If yes, you are welcome to share your workaround and mark it as a solution so that other users can benefit as well. If you find a reply particularly helpful to you, you can also mark it as a solution.
If you still have any questions or need more support, please feel free to let us know. We are more than happy to continue to help you.
Thank you for your patience and look forward to hearing from you.
Hi nilendraFabric ,
Based on your reply does it mean that we have to restrict our data when using direct lake? Shouldn't the date filter help in framing data for PBIX?
What would you suggest is the best way to leverage direct lake for this massive data set? It feels like it defeats the whole purpose of unified data platform for high volume data when it comes to reporting using direct lake
Although Direct Lake doesn’t support model table partitions directly, you can partition your data at the Delta table level:
• Partition your 1.3 billion row fact table by date ranges, regions, or other logical segments
• This can help reduce the amount of data that needs to be loaded into memory for specific queriesThanks again and sorry i forgot to mention, data is sitting in Fabric Data Warehouse hence partitioning isn ot an option.
