All Dataflow posts

Re: [BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

sahillihas20 — Mon, 07 Sep 2026 19:33:46 GMT

Hi Chaithanya, Murtaza,

Apologies for the delayed update — wanted to close the loop here since you both helped investigate this.

We had a support call with (Microsoft Customer Support) on the official ticket, and the issue is now resolved. Sharing the findings here in case it helps others hitting the same error:

ROOT CAUSE (confirmed by Microsoft):
The 404 was a cross-tenant authentication/context issue, not a Fabric backend routing bug as we initially suspected. It was reproducible for both external (guest) users and the internal Workspace Admin, but for different underlying reasons:

1. External (guest) users — needed explicit access granted to the underlying Fabric connection used by the Dataflow, in addition to Dataflow edit permissions. Once connection-level access was granted, external users could edit the Dataflow successfully.

2. Both internal and external users — using "Sign-in Options" during destination connection creation and explicitly selecting the correct tenant resolved the 404 immediately.

3. Internal user (Workspace Admin) — the issue also resolved after clearing the browser session/cached credentials and re-authenticating, without needing explicit tenant selection. This suggests cached tenant-context from a prior session was contributing to the failure.

Microsoft's assessment: this aligns with expected design for cross-tenant access scenarios — external users must explicitly authenticate against the correct tenant when creating a new connection, for tenant isolation/security reasons.

MURTAZA — your hypothesis about it being a backend/API routing issue was a very reasonable read given the symptoms (Lakehouse visible in catalog, 404 only on destination selection). Turned out to be tenant context/auth-cache related rather than the API endpoint itself, but your troubleshooting direction (checking ports/firewall rules, confirming with Microsoft on the exact API call) was exactly the right diagnostic approach.

I did also raise a note with Microsoft that for internal users on their home tenant, having to use "Sign-in Options" explicitly (rather than it working via normal OAuth/SSO) still feels like a gap — they mentioned they'd look at escalating that experience to the product team.

Thanks again both for your time on this — really appreciate the responsiveness from the community.

Best regards, Sahil Singh

Re: [BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

v-kathullac — Mon, 07 Sep 2026 12:33:46 GMT

Hi sahillihas20 ,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?

Regards,

Chaithanya

Re: [BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

Murtaza_Ghafoor — Sun, 06 Sep 2026 10:16:03 GMT

sahillihas20

This looks like a Fabric service/API issue rather than a permissions or authentication problem, especially since the same 404 occurs for Admin users and both authentication methods.

The fact that the Lakehouse appears in the OneLake catalog but the destination picker calls api.powerbi.com/.../lakehouses and returns 404 (UK region) is a strong indicator of a backend/API routing issue.

I would recommend:

Standard Public APIs & UI Portal: Port 443 (HTTPS) is required for all root domain endpoints (such as *.fabric.microsoft.com and *.powerbi.com).
If all ports are unblocked and there is no explicit firewall rule implemented thenI Would suggest raise a Microsoft Fabric support ticket with the Session/Activity IDs.

Include the exact failing endpoint and UK region.
Ask Microsoft to confirm whether the Dataflow Gen2 destination picker is incorrectly calling the Power BI API instead of the Fabric API.
I would avoid relying on the Git queryMetadata.json workaround until Microsoft confirms the supported CI/CD schema for destinations.

If you can attach the relevant screenshots, I can also review them for any additional clues.

If this helps, ✓ Mark as Kudos | Help Other

Re: [BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

sahillihas20 — Fri, 04 Sep 2026 16:40:42 GMT

Hi Chaithanya,

Thankyou for the inputs !
Sure, have raised the support ticket with the required details.

Thanks & Regards, Sahil Singh

Re: [BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

v-kathullac — Thu, 03 Sep 2026 06:11:44 GMT

Hi sahillihas20 ,

Based on the troubleshooting performed, this appears to be a Fabric service-side issue with the Dataflow Gen2 Lakehouse destination configuration in the UK region, rather than a permissions or authentication issue.

The Lakehouse is successfully visible in the OneLake catalog, but selecting it triggers a 404 from the downstream Power BI Lakehouse API.
The issue reproduces for all users, including Workspace Admin, and both Organizational Account and Service Principal authentication were tested.
Since the failure now occurs before the authentication dialog is displayed, the issue appears to be occurring earlier in the destination configuration flow.
The queryMetadata.json modification is not a reliable workaround because Fabric removes the manually added destination properties during synchronization.
For CI/CD, the supported approach to investigate is the Dataflow definition / mashup.pq DataDestinations configuration, rather than manually modifying queryMetadata.json.

At this point, we recommend escalating this to Microsoft Support as a potential UK-region backend/API issue. Please provide them a Session ID, Request ID, Workspace ID, Lakehouse ID, Dataflow ID, timestamp, and the 404 error details so Microsoft can investigate the downstream API call.

Below is the link to create a Support ticket:

Create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn

Regards,

Chaithanya.

[BUG] Dataflow Gen2 destination picker 404 — api.powerbi.com lakehouses UK region

sahillihas20 — Wed, 26 Aug 2026 23:16:25 GMT

Hi community,

We have a complete blocker with Dataflow Gen2 (CI/CD) destination configuration in our UK-region Fabric tenant.

The Error:
When clicking "Add default destination" and selecting a Fabric Lakehouse from the OneLake catalog, this error fires immediately for ALL users including the
workspace Admin:
DataSource.Error: Downstream service call to url 'https://api.powerbi.com/v1/workspaces/{WORKSPACE_ID}/lakehouses' failed with status code 404. (Region: uk)

The Lakehouse IS visible in the OneLake catalog — it appears in the picker. The 404 fires the moment it is clicked.

Root cause hypothesis:
The destination picker calls the legacy Power BI REST API: api.powerbi.com/v1/workspaces/{id}/lakehouses ← returns 404

For Fabric-native workspaces in the UK region, the correct endpoint should be: api.fabric.microsoft.com/v1/workspaces/{id}/lakehouses

The OneLake catalog listing (which uses a different path) resolves the lakehouse correctly — it's only the destination configuration call that uses the wrong endpoint.

Workarounds attempted — all failed:
1. UI — Admin user attempted same steps → same 404
2. queryMetadata.json via Git — Added destination fields, pushed to prod branch → Fabric silently strips loadEnabled/destination on every sync, creating a permanent conflict loop (workspace ≠ Git on every update)
3. Authentication methods — Before the error began firing immediately, a "Connect to default data destination" screen appeared offering two authentication options for the Lakehouse connection:
- Organizational account (Sahil Singh / user account)
- Service principal

Both were attempted. Neither succeeded — the 404 followed in both cases.

Subsequently, this connection credentials screen stopped appearing altogether. The 404 now fires immediately upon clicking lh_dhl_prod in the OneLake catalog, before the connection setup dialog is even shown. This regression suggests the failure point has moved earlier in the destination configuration flow.

Questions:
1. Is this a known issue for UK-region tenants?
2. Is there a way to configure the Dataflow Gen2 destination via Fabric REST API directly to bypass the broken UI?
3. Are there other file-based approaches (mashup.pq metadata) for CI/CD Dataflow Gen2 destinations?

Happy to provide Session IDs and further details to the Fabric team.

I have attached the relevant images, PFA...

Thanks

Re: iifnull() failing on non-null input

miguel — Wed, 19 Aug 2026 05:10:13 GMT

understood! in this case I'd recommend posting this question in the Azure Data Factory community forum from the link below:

Azure Data Factory | Microsoft Community Hub

If you are using a Fabric, you can also migrate your Mapping Dataflows into a Dataflow Gen2 following the guide below:

Mapping Data Flow Transforms in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn

Feel free to post any new questions if you require any help within the scope of Fabric Data Factory and Fabric Dataflows.

Re: iifnull() failing on non-null input

cmjcf — Wed, 19 Aug 2026 04:59:34 GMT

This is a "Data flow" in non-Fabric ADF. The JSON representation has the following:

"type": "MappingDataFlow"

Re: iifnull() failing on non-null input

miguel — Mon, 17 Aug 2026 07:01:09 GMT

Hi!

would you mind clarifying what type of Dataflow you’re using?

iifnull() failing on non-null input

cmjcf — Mon, 17 Aug 2026 03:40:08 GMT

The documentation for iifNull() says this:

Returns the first not null item when given two or more inputs.

We have a dataflow which handles the contents of a number of files, using a specified CSV format. One of the columns is named 'Dealer'. Due to an error in an upstream system, a significant number of files came in with the header misspelled 'Delaer'. There are so many of these files that we cannot simply edit them manually to fix. I have the following expression in a Derived Column transform:

iifNull('Dealer',byName('Delaer'))

I understand this expression to mean "Use the value of the column 'Dealer' if it is not null, else use the value of the drifted column 'Delaer'." (That name is not in the list set by the dataset, so it is drifted, and leaving out byName() causes validation to fail.)

When the file does not contain the spelling error, my expected result is that it simply retains the correct value in 'Dealer' because this is not null. However, when actually run, this expression actually returns null, and causes the column in the Derived Column transform to not be created, resulting in a "Resolved attribute(s) [name] missing from [names]" error.

Re: Dataflow Gen2 - failing to load into the lakehouse

Murtaza_Ghafoor — Fri, 14 Aug 2026 07:02:20 GMT

gabrifly

This is generally a lakehouse name and query in dataflowGen2 mismatch, as an example if your query name is

As an example if your destination lakehouse name

Custom_ABC

Query Name 

Custom-ABC or Custom.ABC

This looks Microsoft Community case where the reported error was exactly this Lakehouse naming error, and the resolution involved the Dataflow entity/query names, not merely the physical source table.

Further look for column names those generated dynamically from ADF ---->Dataflow---->Lakehouse , although you did check the column names but perfroming further checks could be beneficial.

What i mean to say whether the schema entering WriteToDataDestination is deterministic or non deterministic.

At last you can create a test case where you create only table and run the same process from there you debug the actual and isolate the actual problem, and do remember to create new lakehouse where you perform the test, this will narrow down the actual problem, thnaks

If this helps, ✓ Mark as Kudos | Help Other

Re: Data transform using power query

Devvver — Thu, 13 Aug 2026 20:36:05 GMT

Has the problem been resolved or not?

Re: Dataflow Gen2 - failing to load into the lakehouse

v-saisrao-msft — Wed, 29 Jul 2026 05:11:08 GMT

Hi Mauro89,

Thanks for confirming. Since you've already verified the documented naming requirements and the failures are intermittent across different tables without any query changes, this doesn't appear to match a typical naming validation issue. As an additional isolation step, you could test writing the same Dataflow Gen2 output to a new Lakehouse or a new destination table to determine whether the behavior is specific to the current destination. If the issue persists, I'd recommend opening a Microsoft Support case and providing the captured Request ID, as the available documentation doesn't describe this intermittent WriteToDataDestination/Lakehouse036 behavior and further investigation may require backend diagnostics.

Create a Fabric and Power BI Support Ticket - Power BI | Microsoft Learn

Thank you.

Re: Dataflow Gen2 - failing to load into the lakehouse

Mauro89 — Tue, 28 Jul 2026 12:41:54 GMT

Hello gabrifly,

I have some additonal troubleshooting hint what you can try.

- check the namings of the source (ADF) which is taken bei the DF Gen2. Maybe it refers to some naming changes from there rather than in the destination (assuming by experience that some of the error messages are not quite "helpful").

- check if the DF Gen2 was updated, not only the sources or destinations

- check the data source authentication and if for the failed runs in might correlate with the authentication account (e.g. if you authenticate with a user which is bound to PIM activation this could also lead to issues. again not refering to the actual error message as being helpful)

- check what happends if you define a new destination for the same data like into another Lakehouse, does with work consistently?

Hope this helps troubleshooting.

If so, leave some Kudos or mark the post as solution.

Best regards!

Re: Dataflow Gen2 - failing to load into the lakehouse

gabrifly — Tue, 28 Jul 2026 07:24:09 GMT

Hello, thanks for taking the time to answer. I already verified that the naming rules are respected. Also, if naming issues were actually behind the failures, I would expect a consistent pattern where the refresh fails everytime. But since this is intermittent, without any edit in the queries in between, I am inclined to rule out the naming. I have the request ID written down but I could not find any error logs besides the one reported in my post.

Re: Dataflow Gen2 - failing to load into the lakehouse

v-saisrao-msft — Mon, 27 Jul 2026 04:39:15 GMT

Hi gabrifly,

Have you had a chance to review the solution we shared earlier? If the issue persists, feel free to reply so we can help further.

Thank you.

Re: Data pipeline failed. How do you troubleshoot?

v-aatheeque — Fri, 24 Jul 2026 12:58:04 GMT

Hi rajan889262

Following up to confirm if the earlier responses addressed your query. If not, please share your questions and we’ll assist further.

Re: Dataflow Gen2 - failing to load into the lakehouse

v-saisrao-msft — Fri, 24 Jul 2026 09:43:29 GMT

Hi gabrifly,

Since the failure occurs during WriteToDataDestination, verify that the destination table (derived from the query name) complies with the supported naming rules and review the destination mapping. Also, review the Lakehouse destination settings to ensure the expected Replace and Dynamic schema behavior is being used and that no recent schema or destination configuration changes have occurred. If the issue continues intermittently across different tables despite valid naming and unchanged configuration, collect the Request ID and refresh history to help correlate the failures and continue the investigation.

Dataflow Gen2 default destination - Microsoft Fabric | Microsoft Learn

Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn

Thank you.

Dataflow Gen2 - failing to load into the lakehouse

gabrifly — Thu, 23 Jul 2026 12:39:15 GMT

Hello. I have a dataflow gen2, fetching data from an azure data factory environment and loading them into a Fabric Lakehouse. It has been working flawlessfly for the last 3 months, but since two weeks, on some of the tables, I get this error:

"There was a problem refreshing the dataflow: "Something went wrong, please try again later. If the error persists, please contact support.". Error code: EntityUserFailure. (Request ID: 6d111afc-11ee-459f-9e31-db4d08c32c44)."

"XXX - ABC Entity Management_WriteToDataDestination: There was a problem refreshing the dataflow: "Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Error in creating lakehouse table in a version, InnerException: Tables can only contain uppercase or lowercase letters, numbers or the underscore character., Underlying error: Tables can only contain uppercase or lowercase letters, numbers or the underscore character. Details: Reason = Expression.Error;ErrorCode = Lakehouse036;Message = Tables can only contain uppercase or lowercase letters, numbers or the underscore character.;Message.Format = Tables can only contain uppercase or lowercase letters, numbers or the underscore character.;ErrorCode = Lakehouse019;InnerErrorCode = Lakehouse021,Lakehouse019;Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: 1ee1c1a1-a11v-32bt-a19b-5ff6d874f432". Error code: 104100. (Request ID: 66d111afc-11ee-459f-9e31-db4d08c32c44)."

The weird part is that there is no consistency between which tables go to error and which ones proceed to being loaded. Sometimes it works sometimes it does not. Also, there are no issue with the naming, as all the columns are respecting the requirements laid out from the error. And above all, it was working flawlessly until few weeks ago and nothing a

Re: Data pipeline failed. How do you troubleshoot?

v-aatheeque — Tue, 21 Jul 2026 12:14:21 GMT

Hi rajan889262

Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.