Forum Discussion
Memorable MemberTables missing when connecting to dataflow from PBI desktop (2023)
It is a little buggy. Try this:
- Set all 5 tables to load. Can you see them in Power Query?
- If not, go to the tables you cannot see and add a column. Doesn't matter, just have the column be 1 or something.
- Save and let the dataflow process.
- Can you see them now? You should.
- Go back and delete those fake columns, and let it refresh again.
That is what I've done a few times this past week with dataflows that look ok in the service but cannot be seen by Power Query.
The following assumes you aren't a Microsoft employee, and you don't already know this information.
I went ahead and reported the bug, and they were aware. They seemed to acknowledge that at a least one other customer had already reported it. For me the problem happens often enough that it seemed extremely severe (eg. at least once a day my dataflow schemas would become corrupted, forcing me to re-open, edit, save).
Here are details about the CSS support case, and underlying bug. Microsoft wanted to anonymize the details for some reason, so I'm suppressing the first or last of the digits of all the identifiers. However Microsoft assures me that, by using the identifiers and their descriptions, they should be able to retrieve the bug.
Customer Support case 23030300400XXXXXIssue Definition
Schema missing for a table in a dataflow
Summary and resolution
--In power bi service, we have created a dataflow with three tables. After creating the dataflow, we have an issue while trying to view the schema of one table
--But after following the workaround in the community link by adding a column >>Save >>Remove column >>Save. You were able to see the schema of all the three tables.
-- Internal ICM chat-ticket (ICM: XXX174745 – Table schema is missing for some tables in a dataflow)
-- We got an update from the product team that this is a bug (Bug id: XXX4610 – Prevent Object Model’s “Attributes” field from being null) and fix will be deployed in the next few weeks where now it is in testing stage.
...
While Microsoft claims the bug fix will be deployed in the next few weeks, I have some doubts about the likelihood that everything will be peaches and cream. For one, the bug was pretty severe, based on my experiences, and it should have been fixed long ago (2019). Given the bug hasn't been fixed, it implies that a permanent fix may be hard to come by. Secondly I still have a "blue question mark" in the dataflow authoring experience, and it rarely goes away in a timely manner. I'm guessing that this icon indicates an underlying problem with the evaluation of dataflow entities (something that also needs to happen when saving the dataflow as well). If the authoring experience suffers from "blue question marks", then saving the dataflow is also unlikely to succeed. Its just a hunch, although CSS says the "blue question mark" is not relevant.
Hopefully the PG will surprise us, and find a way to permanently fix the bug. However I also have several other dataflow bugs, and none of the bugs ever seem to be fixed very quickly so I'm not holding my breath on this one either. My vnet-gateway bugs have been biting for a couple of years.
Based on the interactions with CSS, Microsoft agreed with the proposed workaround (add a column, save, remove a column, save). They also encourage customers to re-open this ticket if the same problem persists after a month or so.
So it's now 2025. We are still seeing these issues (Version: 2.139.1678.0 64-bit (January 2025)). For us it is also affecting our ability to refresh any downstream dataflows or semantic models. So I fixed it yesterday by adding a column (to a table I coudn't see in PowerBI desktop, I don't know if that mattered), saving, removing and saving it again. Then the refresh worked. But it failed to refresh again today. So when is there a longterm fix coming for this issue?
dbeavon3Heijtink
Gen1 or Gen2? I'm assuming Gen1?
Since the problem was known back in 2019, I think the time for a "longerm fix" has come and gone. (at least for any Gen1 problem).
I would suggest recreating the problem within the context of the more expensive Gen2 dataflows. If it still happens then open a support case that specifically refers to those dataflows. Microsoft is probably more motivated to help with their new and more pricey "Gen2" dataflows.
(FYI, I wanted to mention that in 2024 there were serious bugs which they have refused to fix in Gen1 dataflows, and they told me to use Gen2 as my solution. They were security issues related to oauth token timeouts. As such, you should probably not waste time on any support ticket about Gen1. Just a fair warning. I don't love Gen2 either, because of the cost. It is what it is, unless you want to transition to your own python notebooks with pipelines or something along those lines.)Yeah, Gen1. We were talking about it today, and this was the thought that also popped in my head. I suggested to leave a response here and at the same time move to gen2 dataflows. So far the migration has halted. I just exported the gen1 dataflow and imported it when creating a gen2 but I have nothing but trouble refreshing once published. The queries seem to work fine when testing them in the editor. But once published and then during refresh we get:
Error Code: Mashup Exception Expression Error, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Failed to insert a table., InnerException: We cannot apply operator & to types Number and Record., Underlying error: We cannot apply operator & to types Number and Record. Details: Reason = Expression.Error;Message = We cannot apply operator & to types Number and Record.;Detail = [Operator = "&", Left = 2, Right = [...]]Even when we build it entirely from scratch (copy & pasting the queries). What seems unclear to me is whether I must select a destination or not.
- dbeavon3
>> nothing but trouble refreshing once published.
Yes, gen2 is almost as troublesome as gen1 (but more costly). On a more positive note, you can look forward to some things:- control over "enable staging"
- oauth token refresh so that PQ can exceed an hour
- data sent to other file formats (internally), rather than just dumping it all in CSV text files.
>> So far the migration has halted.
Makes sense. If it was me, I would NOT do a wholesale migration to gen2. No good reason for that. The bug here was relatively easy to work around.However I would plan to build gen2 stuff in the future for new work. And if we are lucky, maybe Microsoft will eventually have pity and start decreasing the cost of gen2 so that it is comparable to gen1!