Forum Discussion
Data Model in Service duplicating and replacing rows
Hi Ed. As I said over email, we've identified the root cause of this issue. We're currently targeting the fix for our July release.
In the meantime, you can work around the issue by ensuring that your folded queries are always sorted. This can be done by always including (and preserving in downstream queries) all primary keys for a given table. Or, if this isn't possible (as in your case when doing a Group By operation), you can explicitly sort by a set of unique keys before combining. (Also note that it may be necessary to buffer after sorting and before combining to ensure the sort is applied by the folding layer. You'll have to experiment to see whether or not this is necessary.)
Ehren
Hi all. Do you ever see duplicated rows after refreshing locally in PBIDesktop, or only when refreshing via the Gateway?
Ehren
Only via the gateway. I have the June 2018 version 14.16.6724.2 installed, which is what https://www.microsoft.com/en-us/download/details.aspx?id=53127&751be11f-ede8-5a0c-058c-2ee190a24fa6=True is pushing. (it shows a different version there, but it sends the version I listed.)
Ok. Does the duplication only happen for rows coming from a SQL Server data source? Or does it happen for other data sources as well?
Ehren
Most of our data is from SQL Server, so I've only seen it happen there.
Edit: Actually, 100% of our data through the gateway is SQL Server. In the few cases where we use CSV files or something else, that isn't via the gateway. I'm not mashing up any gateway+non-gateway data.
I ran a test and this is what I am seeing:
- The table that shows the error is a result of a Table.Combine in Power Query.
- I took the two source queries to that and loaded them into the data model.
- I then used a DAX UNION() statement to generate a new table in Power BI.
- I build a new measure in the new DAX table to replicate a mesaure in the table loaded from the Table.Combine query and stuck it on a card.
- Refreshed everything in Power BI Desktop. Both the measure from the orginal table and the measure from the DAX UNION function show the same result.
- Published to my workspace, connected to the Gateway and refreshed.
- The measure in the DAX UNION table works fine.
- The measure in the original table is off by these random rows.
Something isn't right with how the Table.Combine statement is being loaded either by the Gateway or being handled by the service. The desktop app is fine.