Data Model in Service duplicating and replacing rows

[edhans]

I have a ticket open with MS on this but am interested to see if anyone is noticing an issue where their reports in the Service from published PBIX desktop files are not right. Here is what I've found:

• The record count is always correct. For example, we have 877 records in May 2018. That is correct.
• Some records are duplicated, but something in the data model is telling it to maintain the same record count, so other records are removed. This is 100% in the data model (DAX) portion. Not the query.
• I've seen reports refresh ok (most of the time) and then randomly exhibit this behavior.
• Started late last week. (towards the end of the June 11, 2018 week)
• I have verified there is no corruption in the PBIX file itself as I redownloaded it from the service and refreshed it on my desktop. The numbers work.
• If I download the PBIX from the service when it shows wrong numbers, and don't refresh it I can pinpoint which rows are duplicated and which are removed by using the new filter views in the PBI Desktop app and copying the filtered tables to Excel for further analysis. THat is how I know the record count is always correct.

As I said, I have a ticket open and have started the initial conversations, but wanted to see if anyone here was experiencing the same issue recently.

 On the June 2018 PBID and June 2018 Gateway. These are SQL Server tables. I've not seen this issue on other data types, but most of our data is SQL Server.

[Ehren]

Ed, thanks again for your time today. I'll keep you posted as we continue to track down the root cause of this issue.

Ehren

[Ehren · Accepted solution]

Hi Ed. As I said over email, we've identified the root cause of this issue. We're currently targeting the fix for our July release.

In the meantime, you can work around the issue by ensuring that your folded queries are always sorted. This can be done by always including (and preserving in downstream queries) all primary keys for a given table. Or, if this isn't possible (as in your case when doing a Group By operation), you can explicitly sort by a set of unique keys before combining. (Also note that it may be necessary to buffer after sorting and before combining to ensure the sort is applied by the folding layer. You'll have to experiment to see whether or not this is necessary.)

Ehren