Dataflow Gen2 Unable to Write

[Anonymous]

Hello All,

I've been unable to get dataflows to work in two different workspaces. I am trying to load data from Salesforce objects to Fabric. Here are the errors I am recieving. It seems like my instance of Fabric is just broken.. I get errors just trying to open some of the warehouses.

Also, the submit button on the support form doesn't seem to work. I am on the Free trial, is it related to that?

Dataflow name	Dataflow refresh status	Table name	Start time	End time	Duration	Engine	Refresh status	Status details
SF_Account_Dataflow	Failed	NA	2023-08-31T20:17:32.13+00:00	2023-09-01T04:23:15.97+00:00	8:05:43	NA	Failed	There was a problem refreshing the dataflow. Please try again later. (Request ID: 49104a42-3a60-40eb-a2d1-4ce3ea19d784).
SF_Account_Dataflow	Failed	Account	2023-08-31T20:17:41.647+00:00	2023-08-31T20:23:05.013+00:00	0:05:23	NA	Succeeded	NA
Dataflow name	Dataflow refresh status	Action	Start time	End time	Duration	Engine	Refresh status	Status details
SF_Account_Dataflow	Failed	WriteToDatabaseTableFrom_TransformForOutputToDatabaseTableFrom_Account	2023-08-31T20:23:05.0295499+00:00	2023-09-01T04:23:06.4325146+00:00	8:00:01	NA	Failed	Timeout Entity refresh timed out. Please try again or consider reducing the size or complexity of the entity and try again.

[BryanCarmichael]

Gen2 Dataflows are still not usable in a production environment as they still have a lot of issues on the load side of their operation. 

We are actively engaged with Microsoft Engineers playing whack-a mole and in fairness a lot of progress has been made.

In the mean time we have adopted the follwing solution 
1: Load to staging datamart
2: Create a data pipeline that first executes a delete stored procedure on target table in warehouse, then loads data from datamart to datawarehouse using a copy data operation

it is a bit of a faff to setup but once built works well with no intermittent failures that can be exeperienced with Gen2 Dataflows

[HimanshuS-msft]

Hello @Anonymous 
Thanks for using the Fabric community.

Looking at the error "Timeout Entity refresh timed out. Please try again or consider reducing the size or complexity of the entity and try again." , sounds like either its s momentary issue or the model is really complex . Since you are using a trial version which has a 64 CU alloted . I suggest you to try with something more simple and see if that works . 

Let me know how it goes .

Thanks
HImanshu