DataFlow with DataWarehouse in other Workspace Error
Hello, I have a dataflow in one workspace that I want to configure with a DataWarehouse destination in another workspace. However, I have the following warning: "Due to performance reasons, only Warehouses in the current workspace are shown." Has anyone had this problem before? Normally I should be able to write to any Data Warehouse (it doesn't matter which workspace, as long as I have the rights). Have a nice day,
connecting dataflow to a lakehouse folder of files to combine them
I want to use dataflow gen2 to process a folder full of CSV files (all identical in structure) and load the combined and cleaned data in lakehouse table. The folder is in a lakehouse in the same premium workspace as the dataflow. miguel in Solved: Re: Can Dataflow Gen2 read from lakehouse "Files" ... - Microsoft Fabric Community says "You can use the combine files experience the same as with other file system views (SharePoint, data lake, folder and others). " Therefore I'm expecting to see a "Combine" button when i connect to the folder, but I do not. Am I doing something wrong? Details. 1. New dataflow gen2 2. Get data> More 3. OneLake data hub 4. Click on the lakehouse 5. browse to the folder. Note I can select files individual, or stay on the folder name. The only option for the folder is "Create", as shown: If I choose Create I get a table with a list of the file contents, as shown: Once I have that, I can drill down into one row at at time to get the csv file, but not sure how to process all of them efficiently (there will be dozens in production). Any advice, other than using a notebook instead .
Dataflow Gen2 Mashup Error: Loading Data into Lakehouse
Hi all, I'm currently doing this fabric tutorial on youtube (https://www.youtube.com/watch?v=HLVkIsZvgzE). Uploading the parquet files into the Lakehouse was no problem. Then one needs to convert them into tables. With some of the files this is no problem however others have spaces in the column names which cause an error and cannot be convertet directly into a tables. For this the video shows a method how one can use dataflow Gen2 to convert the column names. After setting up the dataflow Gen2 and publishing it as shown in the video on 23:00 min I get the following error: Data_Store_1 parquet_WriteToDataDestination: Mashup Exception Data Format Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataFormat.Error: Error in replacing table's content with new data in a version: #{0}., InnerException: We can't insert values of type '#{0}' into column '#{1}' because it expects values of type '#{2}'., Underlying error: We can't insert values of type 'DateTime' into column 'Open_Date' because it expects values of type 'Date'. Details: Reason = DataFormat.Error;Message = We can't insert values of type 'DateTime' into column 'Open_Date' because it expects values of type 'Date'.;Message.Format = We can't insert values of type '#{0}' into column '#{1}' because it expects values of type '#{2}'.;Message.Parameters = {"DateTime", "Open_Date", "Date"};Microsoft.Data.Mashup.Error.Context = User In the youtube tutorial this error does not show up, so does anyone know how I can fix this?? Kind regards
Dataflow gen2 to lakehouse table type error
A dataflow gen 2, last step, conver column type to date. Dataflow run ends with error cant insert dateTime into date Last df step: #"Tipo de columna cambiado" = Table.TransformColumnTypes(#"Remove Columns", {{"FEC_MATRICULA", type date}, {"FEC_TRAMITACION", type date}, {"FEC_TRAMITE", type date}, {"FEC_PRIM_MATRICULACION", type date}, {"FEC_PROCESO", type date}}) Dataflow run error: Mashup Exception Data Format Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: DataFormat.Error: We can't insert values of type 'DateTime' into column 'FEC_MATRICULA' because it expects values of type 'Date'. Details:
