Dataflow Refresh error using incremental refresh

[aledaniel]

Hello,

I need to create dataflows with warehouse movemente from my ERP system. I have the gateway properly installed on a server, create connection to the ERP database (postresql) using odbc connector. It works properly with some datasets I've already upload in PBI service.
The movements are more or less 27million rows. If I try to extract them with a single dataflow I have an error (I suppose it's caused by not enough resources in the ERP database to answer the query or limitation of the odbc connector... I do not know). So I created more dataflows based on years intervals (2000-2004/2005-2009/ and so on) with a limit of 5-6millions of rows (more rows generate the error). On the last dataflow (2019-today) I set up the incremental refresh (10 years in archive and 18 months update. I have put 10 years in archive because I obviously do not want to lose the oldest movements, considering that incremental refresh is "rolling"). The problem is that refreshing this dataflow I have always the following error:
Error: Encountered user gateway exception: '[GatewayPipelineErrorCode]=DM_GWPipeline_Gateway_SpooledOperationMissing [ErrorShortName]=GatewayClientErrorResponseException[GatewayId=639729]/UnknownSpooledOperationIdException [ExceptionErrorShortName]=GatewayClientErrorResponseException[GatewayId=639729]'.. RootActivityId = 2f9cf085-b3dc-4d72-af47-580037a5ac49.Param1 = [GatewayPipelineErrorCode]=DM_GWPipeline_Gateway_SpooledOperationMissing [ErrorShortName]=GatewayClientErrorResponseException[GatewayId=639729]/UnknownSpooledOperationIdException [ExceptionErrorShortName]=GatewayClientErrorResponseException[GatewayId=639729] Request ID: 96caf062-c8f3-8909-27d7-151ba68a33e4.

If I remove the incremental refresh the dataflows refreshes properly.

Anyone can help me with this issue?

[v-yangliu-msft]

Hi  @aledaniel ,

In your scenario, please check that if your account that used to connect to PostgreSQL database has sufficient permissions on the tables you imported into Power BI.

If there is no issue with permission, please delete the data source under gateway and re-connect the data source, make sure connection strings of the data source under gateway are same as that in Power BI Desktop.

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[v-yangliu-msft]

Hi  @aledaniel ,

1. Are you an admin of the gateway you want to use? 

2. Do you have access to use the gateway?

To create a dataflow using a gateway, the user must be the Enterprise Gateway’s administrator, or the administrator must have shared the datasource they are planning to use with the user.

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[aledaniel]

Hello, 

yes I'm admin of the gateway and I have access.
As I explained, the refresh works correctly if I do it manually or if I set a planned refresh. I have got the error if I try to set up incremental refresh.

Thank you for your help.