Forum Discussion

joakimfenno's avatar
joakimfenno
Icon for Helper V rankHelper V
2 years ago
Solved

Fabric - rows per table limitation

I have read about Semantic Model SKU limitation here I am concerned about max number of rows per table in lakehouse and warehouse as well as limitations on semantic model. Is it correct that direct...
capacities
  • AndyDDC's avatar
    AndyDDC
    2 years ago

    300M is the row limit per table in an F8 SKU where DirectLake will be supported.  Any tables above 300M when running on an F8 capacity will fall back to DirectQuery.

     

     

joakimfenno's avatar
joakimfenno
Icon for Helper V rankHelper V
2 years ago

we made some tests and compared a table of 300M rows with one of +300M rows
direct lake mode could not be used for the second one
some of the descriptions below are in swedish but you should be able to understand what teh tables show

 

300M (for F8 capacity in our case) is not a hard limit for storing but for consumption it is significant limitation

 

 

lbendlin's avatar
lbendlin
Icon for Super User rankSuper User
2 years ago

What's the error message? 300M is small, even for a F8

  • AndyDDC's avatar
    AndyDDC
    Icon for Most Valuable Professional rankMost Valuable Professional
    2 years ago

    300M is the row limit per table in an F8 SKU where DirectLake will be supported.  Any tables above 300M when running on an F8 capacity will fall back to DirectQuery.

     

     

    • frithjof_v's avatar
      frithjof_v
      Icon for Community Champion rankCommunity Champion
      2 years ago

      I guess this is related to the fact that Direct Lake loads the entire column into the Power BI Vertipaq cache.

       

      Even if the Power BI end user only queries a subset of the rows, there is no WHERE-filter pushdown to the Delta Tables, only column pruning. So the entire column (all rows) will be loaded into memory.

       

      I guess the row limit is a hard limit in the semantic model memory (Direct Lake Vertipaq cache), with the size of the row limit decided by the SKU size.

       

      Filters in the DAX queries are pushed down to the in-memory vertipaq cache, but filters are not further pushed down to the delta tables. Only column pruning is pushed down to the delta tables. 

       

      I guess also the delta table version timestamp is pushed down as part of the query to the delta tables. This is relevant if automatic refresh (automatic reframing) is disabled. In this case, the vertipaq cache will load the version from the delta tables as determined by the time of the last reframing operation of the semantic model.

       

      It would be cool if it was possible to do time travel in a direct lake semantic model. I.e. let the owner manually pick which versions of the delta tables to use in the direct lake semantic model.

      • AndyDDC's avatar
        AndyDDC
        Icon for Most Valuable Professional rankMost Valuable Professional
        2 years ago

        Nope no filter predicate push down into the delta tables via direct lake, all or nothing. Then filtered in the vertipaq by whatever dax/report filters are in place 

  • joakimfenno's avatar
    joakimfenno
    Icon for Helper V rankHelper V
    2 years ago

    there is no error message but you cant force direct lake mode in a semantic model for example