DataFlow gen 2 not updating the data in my Warehouse
Hey Community, I have this automated pipeline: The DataFlow performs some transformations and dumps data into the Warehouse. And from the warehouse, I create my report. The pipeline ran successfully, however, the new columns I added in my dataflow are not reflected in my warehouse. Why is this happenning? And how can I make sure the data reflects? Cause I'm getting a lot of data and want it to see the warehouse and dashboard updated. Thanks !
Df gen2 Save Changes deficiency
miguel A major diffrence between gen1 and gen2 is while exiting the gen1 df it gives the following options Gen 1 at exit at bottom vs Gen2 at exit at bottom The implication of the above is, if I open the same gen1 df in two different browser tabs and edit and save one (tabl1 -v1 to v2) and then I can simply exit out of the other dataflow in other browser tab (tab2 - v1) by choosing any of the options (at exit, at bottom). The current datatflow is v2 (desired v2). but for gen2 df if I open the same gen2 df in two different browser tabs and edit and save one (tabl1 - v1 to v2) and then if I exit out of the other dataflow in other browser tab (tab2 - v1) because it neither gives Save/Dsicard/Cancel option at exit nor the Cancel option at bottom, v2 gets overwritten by v1. The current datatflow is v1 (desired v2). It means one would be heavily penalized if unaware of this behavior and can loose hours of work. Can you please test this on your side and let me know if a permanent fix is expected? This has undesirable effects.
Is query folding in data flow related to user credentials access limitations?
I am trying to replicate a Data flow which is created by another person to test out Increemental refresh. I observed that their query steps are folding and my steps are not folding. In general if query is not folding it means that we are unable apply transformations steps directly on the source. Iam referencing following articles. https://learn.microsoft.com/en-us/power-query/query-folding-basics Understanding query evaluation and query folding in Power Query - Power Query | Microsoft Learn Query folding indicators in Power Query - Power Query | Microsoft Learn Please let me know if the following queries makes sense as credentials might be different: Is query folding in data flow related to user credentials access limitations? Ref image: Fold/unfold FAQ on the Fabric User GroupsDataflow Gen2: Data source connection error (API)
mslearn-fabric (microsoftlearning.github.io) I have Data Source connection issues while trying to create Dataflow(Gen2) via Synapse Data Engineering using the guidelines in the link above. The import tabs (Excel, SQL, Text/CSV, dataflows) give the error message below. "Something went wrong while loading required resources. Please try refreshing the page or come back later to try again. (APIs: connections In Scope)." Much more, both import from Excel and Text/CSV tab generated "Unexpected error" page restricting further access to the import pages. Please see error message below. ---------- Message ---------- Unhandled HTTP Request Exception. Status Code: 400. Reason: undefined. Response Text: "ExternalServiceError". ---------- Stack ---------- ApiResponseError: Unhandled HTTP Request Exception. Status Code: 400. Reason: undefined. Response Text: "ExternalServiceError". at p.handleLoadEvent (https://uk.prod.powerquery.microsoft.com/MashupEditor/scripts/p.h.a.c2dedd539a7896950b5f.bundle.js:2:34257) at XMLHttpRequest.<anonymous> (https://uk.prod.powerquery.microsoft.com/MashupEditor/scripts/p.h.a.c2dedd539a7896950b5f.bundle.js:2:35587) ---------- Session ID ---------- e58577fb-a25f-4462-9228-a9742e974b16 ---------- Request ID ---------- f52b4c18-1052-464d-93eb-7b381f53fe01 Please advise how to resolve this issue.
SQL Analytics Endpoint - Table constraints disappear after running Dataflow Gen2
Hi, I created a Lakehouse with 2 tables: Fact table: RandomNumbers Dimensions table: Dim_Date I created a relationship between the tables in the SQL Analytics Endpoint When I query the [INFORMATION_SCHEMA].[TABLE_CONSTRAINTS] view, I can see that these relationships have been registered as constraints on the tables in the SQL Analytics Endpoint. When I connect to the RandomNumbers table of the SQL Analytics Endpoint from Power Query in Power BI Desktop, I can see that there is an expandable column so I can easily expand columns from Dim_Date into the RandomNumbers query. I have a Dataflow Gen2 which, when it runs, does a Replace with Dynamic schema of the Dim_Date table. After running the Dataflow Gen2, these are my observations: The relationship still appears in the model pane of the SQL Analytics Endpoint ✔️ The relationship still works in the Default Direct Lake semantic model ✔️ The constraints are not found anymore when I query the [INFORMATION_SCHEMA].[TABLE_CONSTRAINTS] view ❌ The expandable column is not found anymore in Power Query in Power BI Desktop. An import mode semantic model I built which uses this expandable column, get failed refreshes because the column can't be found anymore. ❌ My questions: - Is this the expected behavior? - Shouldn't the table constraints and the model relationships in the SQL Analytics Endpoint be automatically synchronized? Thank you! 😀
Dataflow runs since hours, cannot be stopped?
Hello, we are encountering more and more issues with Fabric, unfortunately. The newest addition is a Gen2 Dataflow that runs now since 7 hours while usually it took about 1 hour to complete. I understand there is not cancelling option as of now, how can we stop the Dataflow from running and restart it? Also, when is the option to cancel refreshes released?Dataflow Data Destination - Stuck in Connect to data source screen
Hi everyone, we are performing various tests with Fabric to see if it can be used for our clients. I will try to document the issues that we are encountering on the way. The topic of this post is, that we are stuck in the screen as shown in the screenshot attached. What is the problem here? No matter how many times I close and re-open the dataflow, the problem does not go away.Need Help with Formatting DateTime in Azure Synapse Data Flow Expression Language
Hi All, I'm working with Azure Synapse Analytics and I'm facing an issue with the Data Flow expression builder. My goal is to format a column named "Start Date (BST)" which currently has the format yyyy-MM-ddTHH:mm:ss.SSSSSS to dd/MM/yyyy HH:mm:ss, effectively removing the milliseconds. Here's what the date looks like right now: 2022-07-07T23:43:13.722007 I tried using the following expression in the data flow expression builder: toTimestamp('Start Date (BST)', 'dd/MM/yyyy HH:mm:ss:SS') However, I am getting an error stating "Unable to parse the expression. Please make sure it is valid." Could someone please guide me on the correct syntax or the steps I should take to format the datetime values properly using the expression builder in Azure Synapse? I would appreciate any insights or examples of the correct expression to achieve this. Thank you in advance!
