Hi,
We have a Data Model that is using Direct Query connection to source database- Azure Databricks. The table A and B are joined with one to many relationship in the model on the unique ID. As the data is huge we have partitioned both the table in Databricks based on month . It is noticed in the Databricks query log Partitions are only respected for one side of the table where as for the many side of table all rows (each day) are scanned even after passing a month filter through Power BI. On checking the query execution in Databricks, we see due to the join on ID , table B scans all rows first and then filters on month. This results in performance issue. Is there any way the query execution be modified so that table B respects partition ? For ex if Month from table A is filtered for January, then table B should only hit January partition rather than scanning all rows in Database.
