Fabric - semantic model performance

In theory, I think the size of the model (in terms of number of tables) should not matter so much.

Because Direct Lake only loads the columns it needs from the model (or rather, the underlying delta tables), when a user interacts with the visuals in the report. This is called paging. The column data gets loaded into memory, which makes subsequent queries faster. However when the reports accessing direct lake data is opened first thing in the morning, the data in the delta table columns need to be loaded into memory, which takes some time. It will be faster when using a fewer number of columns in the visuals on a report page. Possibly, you can also schedule semantic-link in a Notebook to query the most used columns of the semantic model in the morning (pre-warm the direct lake semantic model's columns), before people open the reports. 

Limiting the number of columns used in the visuals on a report page will probably help a lot.

Also, lakehouse (or warehouse) tables with many rows (tens or hundreds of million rows) will probably be slower than lakehouse tables with fewer rows.

https://data-mozart.com/50-shades-of-direct-lake-everything-you-need-to-know-about-the-new-power-bi-storage-mode/

https://www.sqlbi.com/blog/marco/2024/04/06/direct-lake-vs-import-mode-in-power-bi/

Because you said you are using Warehouse as the source, perhaps it helps to run some SQL Select queries against the tables frequently to optimize (compact the underlying parquet files).

https://youtu.be/uKgXKAw00XM?si=TumYZhewI_25GHtr

(this is a great video from the Warehouse product team)

https://blog.fabric.microsoft.com/blog/announcing-automatic-data-compaction-for-fabric-warehouse/

Perhaps some of these general performance considerations for Warehouse also applies:

https://learn.microsoft.com/en-us/fabric/data-warehouse/guidelines-warehouse-performance