DataFlow gen 2 not updating the data in my Warehouse
Hey Community, I have this automated pipeline: The DataFlow performs some transformations and dumps data into the Warehouse. And from the warehouse, I create my report. The pipeline ran successfully, however, the new columns I added in my dataflow are not reflected in my warehouse. Why is this happenning? And how can I make sure the data reflects? Cause I'm getting a lot of data and want it to see the warehouse and dashboard updated. Thanks !
Fabric Lakehouse, Data warehouse and Fabric Pipelines on Ownership
We have fabric lakehouse, Data warehouse and Fabric pipelines in power bi workspace with Power BI Premium Capacity(P1). Upon disabling the user accounts. Unable to query the data warehouse / lakehouse tables via SQL endpoint and fabric pipeline gets failed with Message=Failed to get UserAuth access token. The error message is: AADSTS50057: The user account is disabled. To overcome this, we know the solution is to change the ownership of lakehouse, warehouse and for scheduled pipelines we had to modify the schedule/any other activity so that last modified user gets changed (There are about 50+ pipelines). In case if existing user account disabled. We had to modify the ownership again and modify the pipeline activity. Are there any better solution to this?
DataPipeline - DataType issue
Hello, We are pusing data to the Lakehouse using the DataFlow and once data is avilable to the Lakehouse, we are trying to push the data from Lakehouse to Warehosue using Data Pipeline. so, while runnig the Data Pipeline, Copy activity take the Default datatype as Nvarchar(Max) which is not support by Data Warehouse and pipeline got an error(DataType). Below scrrenshot for your reference:
Microsoft Fabric Data Pipeline + Power BI API Scanner
Hello, I'm looking to set up a data pipeline Fabric, I started by creating a copy data activity to retrieve a list of workspaces. But then I'm stuck, I can't manage to break down this list into packages of 100. Then for each block of 100 call the PostWorkspaceInfo API. Does anyone have any ideas? Best regards, Charline
Need Help with Formatting DateTime in Azure Synapse Data Flow Expression Language
Hi All, I'm working with Azure Synapse Analytics and I'm facing an issue with the Data Flow expression builder. My goal is to format a column named "Start Date (BST)" which currently has the format yyyy-MM-ddTHH:mm:ss.SSSSSS to dd/MM/yyyy HH:mm:ss, effectively removing the milliseconds. Here's what the date looks like right now: 2022-07-07T23:43:13.722007 I tried using the following expression in the data flow expression builder: toTimestamp('Start Date (BST)', 'dd/MM/yyyy HH:mm:ss:SS') However, I am getting an error stating "Unable to parse the expression. Please make sure it is valid." Could someone please guide me on the correct syntax or the steps I should take to format the datetime values properly using the expression builder in Azure Synapse? I would appreciate any insights or examples of the correct expression to achieve this. Thank you in advance!Unable to select lakehouse our warehouse in a pipeline (Copy data)
Hello everyone, I'm currently working on setting up a pipeline to transfer data from my Lakehouse to my Warehouse. However, I've encountered a difficulty: neither the Copy Assistant nor the Copy Canvas display the names of my "houses". I've attached an image below for further clarity: typo *appear Interestingly, after several page refreshes, the Lakehouse name finally appeared. However, when it came time to select the data destination, my Warehouse did not appear on the selection list. Then I refresh the page again and my Lakehouse disappeared from the list. Both Lakehouse and Warehouse are saved in myworkspace. Could this be a bug? If anyone has encountered this issue before, or if anyone has insights on how to remedy this situation, I would greatly appreciate your assistance. Thank you in advance for your time and help!
