dataflow refresh issue

[Bber2000]

Dear support team,

I have a dataflow that is refreshed manually on a monthly period. It contains several millions of records. 

usually it takes several hours to refresh but it works.

those last days I tried 10 times to refresh it but it failed every time.

this is the refresh log :

....,Error: PipelineException: Internal error DirectQueryUnexpectedEndOfStream. . RootActivityId = 3577bc33-d975-4312-a5cf-d11242774218.Param1 = PipelineException: Internal error DirectQueryUnexpectedEndOfStream. Request ID: 7f9a5b56-6509-c934-78e8-9856c64d2dfb.

  What should I do to make it work ?

Regards,

[v-yingjl]

What is the data source of this dataflow in this issue and did it happen in Premium workspace or normal workspace? If in Premium workspace, did it be Premium Gen1 or Premium Gen2.

In addition, since this dataflow was refreshend manually on a monthly period, have you tried to configure a gateway for it to refresh and tried different types of workspaces?

Also, 'It contains several millions of records', whether check that dataflow could refresh successfullu by using the same data source with few rows.

Best Regards,
Community Support Team _ Yingjie Li
 

[Bber2000]

the datasource is a dataflow (PowerPlatform.Dataflows)

this dataflow is stored in a premium gen2 workspace.

i have tried to refresh with and without using the gateway. actually on the gateway, i cannot create a powerbi datasource anymore.

and yes, if i retrieve fewer records from the source, the refresh works.

[v-yingjl]

Since few records could be refreshed successfully, looks like it related to the data size. You can try to enable large data datasets in Premium.

See: Large datasets in Power BI Premium 

Best Regards,
Community Support Team _ Yingjie Li

[Bber2000]

I cannot switch to "large dataset" since this is a dataflow.

[v-yingjl]

Whether consider breaking the data source into a series of dataflows and refresh them in near time range to improve the refresh performance to check.

Similar thread: Dataflow size limit 

Best Regards,
Community Support Team _ Yingjie Li