Forum Discussion
SQL Analytics Endpoint - Table constraints disappear after running Dataflow Gen2
Hi,
I created a Lakehouse with 2 tables:
Fact table: RandomNumbers
Dimensions table: Dim_Date
I created a relationship between the tables in the SQL Analytics Endpoint
When I query the [INFORMATION_SCHEMA].[TABLE_CONSTRAINTS] view, I can see that these relationships have been registered as constraints on the tables in the SQL Analytics Endpoint.
When I connect to the RandomNumbers table of the SQL Analytics Endpoint from Power Query in Power BI Desktop, I can see that there is an expandable column so I can easily expand columns from Dim_Date into the RandomNumbers query.
I have a Dataflow Gen2 which, when it runs, does a Replace with Dynamic schema of the Dim_Date table.
After running the Dataflow Gen2, these are my observations:
- The relationship still appears in the model pane of the SQL Analytics Endpoint ✔️
- The relationship still works in the Default Direct Lake semantic model ✔️
- The constraints are not found anymore when I query the [INFORMATION_SCHEMA].[TABLE_CONSTRAINTS] view ❌
- The expandable column is not found anymore in Power Query in Power BI Desktop. An import mode semantic model I built which uses this expandable column, get failed refreshes because the column can't be found anymore. ❌
My questions:
- Is this the expected behavior?
- Shouldn't the table constraints and the model relationships in the SQL Analytics Endpoint be automatically synchronized?
Thank you! 😀
Oh, I see... My bad. It's not regular Replace I am using. I am actually using Replace with Dynamic schema. I had forgotten that. (I now changed the info to Replace with Dynamic schema also in my previous comments)
I guess the Dynamic schema setting is causing the table to get dropped and recreated.
I still find it inconsistent that the constraints and model relationships are affected differently by this operation.However, for my use, the model relationships are more important than the table constraints. So I like that the model relationships are not being dropped at least.
I just need to beware not to use the expand column option in Power Query for my import mode semantic models, when working with lakehouse tables which are subject to Dynamic schema.
As the expand column option seems to be depending on the table constraints in the SQL Analytics Endpoint, and the table constraints don't seem to get recreated when the table gets dropped and recreated.I would prefer that the table constraints also keep working after running a Dataflow Gen2 with Dynamic schema, similar like the model relationships.
5 Replies
Oh, I see... My bad. It's not regular Replace I am using. I am actually using Replace with Dynamic schema. I had forgotten that. (I now changed the info to Replace with Dynamic schema also in my previous comments)
I guess the Dynamic schema setting is causing the table to get dropped and recreated.
I still find it inconsistent that the constraints and model relationships are affected differently by this operation.However, for my use, the model relationships are more important than the table constraints. So I like that the model relationships are not being dropped at least.
I just need to beware not to use the expand column option in Power Query for my import mode semantic models, when working with lakehouse tables which are subject to Dynamic schema.
As the expand column option seems to be depending on the table constraints in the SQL Analytics Endpoint, and the table constraints don't seem to get recreated when the table gets dropped and recreated.I would prefer that the table constraints also keep working after running a Dataflow Gen2 with Dynamic schema, similar like the model relationships.
Hi frithjof_v
Apologies for the issue you have been facing. Thanks for the feedback. I have forwarded the feedback to the internal team.
Hope this helps. Please let me know if you have any further queries. Glad to help.
Just a quick one before I dive a little deeper, this is a Warehouse (and not a Lakehouse)?
It's a Lakehouse
I am suspecting that the reason is that the Dataflow Gen2 drops and recreates the Lakehouse table when using Replace mode with Dynamic schema.
So maybe that's why the constraints are removed (or rather: not re-applied) from the tables in the SQL Analytics Endpoint?
However, I think it's not consistent with the model relationships still being present and working afterwards.
These are the destination settings in my Dataflow Gen2:
