Forum Discussion
lakehouse table duplicates all rows when added to semantic model
Hi SCWO ,
Thanks for the update.
Good to hear that re-creating the table in the silver layer fixed it. It’s possible that the table had some leftover data or files, and the semantic model picked up the same data more than once.
Re-creating the table gave it a clean state, so the duplicates were no longer there.
Thanks for sharing what worked - it’ll help others as well. Please consider marking the reply as the accepted solution to make it easier for others to find.
Thank you.
Yeah, I ran into this once. In my case, the issue was related to how the table got joined or loaded in the model. First thing I'd check is whether the table was accidentally marked as both import and DirectLake at the same time (some Fabric models can silently do that if the dataset isn't cleaned properly before refresh).
Another thing: if your semantic model has auto-detect relationships enabled, it could have created a loop or many-to-many relationship that's effectively duplicating rows on join. I'd also check in Power BI Desktop model view for hidden relationships or inactive ones that got activated during refresh.
If none of that helps, try creating a brand-new model, link only that table, and see if the issue persists, that usually helps isolate whether it's the data or the model config.
Hi alia12,
thanks for your advices. I tried to check them in my model but I found no error.
The only thing ist to create a brand new model with this table, then it works.
But I would be very curious about how this could happen to avoid it for the future.
v-veshwara-msftCommunity Support
Hi SCWO ,
Thanks for using Microsoft Fabric Community and thanks for confirming that the issue disappears when using a new semantic model. That usually means the problem was with how the original model was set up.
As mentioned by alia12 one possible cause is when a table ends up being both Import and DirectLake mode at the same time. This can happen during schema changes or partial refreshes and may lead to duplicated rows.
If the Lakehouse table is a shortcut to another source like a Warehouse or ADLS Gen2, it might have missing metadata. This can also cause issues during model refresh.
Also, if you're using the SQL analytics endpoint to build the model, there might be a short delay before the latest data is fully available. This can show outdated or repeated rows right after loading.
If using Dataflow Gen2 to load data from the Warehouse into the Lakehouse, there have been earlier reports of similar duplication issues. If "append" mode is used, data can stack up and duplicate. Even in "replace" mode, there are cases where the table doesn’t get fully replaced and rows are copied twice.
Reference: Re: Duplicated Rows In Tables Built By Notebook - Microsoft Fabric Community
Since a fresh model works fine, the issue was likely caused by something in the original model’s metadata or load process.
Hope this helps. Please reach out for further assistance.
If the issue is now resolved, please consider marking the reply as the accepted solution so others can find it more easily.
Thanks.
- v-veshwara-msft
Hi SCWO ,
Just wanted to check if the above response has met your needs. If yes, please consider marking it as "Accepted Solution" to assist others with similar queries. If further assistance is needed, please reach out.
Thank you.