Fabric - warehouse - query error Assertion failed: Invalid Redistribute operation

[joakimfenno]

I have a warehouse view in a warehouse sourced from a delta table in a lakehouse

when trying to query the view using aggregate functions (sum etc.) I receive the error below immediately

other queries seems to work

the lakelouse table contains 50M rows and the view is pivoting the data

I can think of mutiple solutions (partitioning, persisting etc.)  but I want to understand the reason for the error

Request to perform an external distributed computation has failed with error "Assertion failed: Invalid Redistribute operation. Could not produce move enforcer

[joakimfenno]

sure, what do you need to know?

the source view are select * from lakehouse.table

some views works fine but a more complex (pivot/cte) (possibly in combination with more data) views does not work either from sql endpoint or semantic model

[Anonymous]

HI @joakimfenno,

After I research on the error messages, I find the message normally appears when the system is unable to properly distribute data across nodes during a query execution. (these often due to complex operations like pivoting or using aggregate functions on large datasets)

For this scenario, you can try to partition data to improve distribution and performance and simplify the query to reducing the number of complex operations (e.g., pivot, CTEs).
Also, you can consider to using intermediate steps instead of directly apply all operations in one query. (store intermediate results in tables to reduce the complexity of the final query)

Regards,

Xiaoxin Sheng

[joakimfenno]

thanks, it worked after persisting the data

do you know is this is somehow related to the size of your fabric capacity?

[Anonymous]

HI @joakimfenno,

I find a document that list the model memory size limit between different SKU, perhaps you can take a look at the following link if it helps:

What is Power BI Premium? - Power BI | Microsoft Learn

Regards,

Xiaoxin Sheng

[Anonymous]

HI @joakimfenno,

Can you please share some more detail information about this issue? They should help us clarify your scenario and test to troubleshoot.

In addition, you can also take a look at the following document about data warehouse limitations:

Limitations - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng