Error on dataflow refresh

[BBConsultancy]

Hi All,

I use a dataflow (gen1) in the Power BI service to refresh the data for one of my customers. I use a dataflow because the customer has multiple data sources. All data flows are used in one sematic model, which is used for the Power BI reports. I only do simple transformations on the dataflow directly:

- Filtering

- Deleting columns

- Assigning data types

Sometimes I get an error during the refresh of the dataflow and it's always on the same table: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

The table has around 600.000 rows. It's the ItemsIncremental tabel from Exact Online. I use the Invantive connector for Exact online. When I check the error in Invantive App it says: The data download was cancelled after 6 minutes, 57 seconds, such as by the user navigating away in the Power BI or Power Query User Interface.

Microsoft's documentation says that the max refresh time for a dataflow is two hours, but this dataflow only takes 15minutes to refresh usually.

This page gives some more info on Power Query Online limitations, but i'm not sure how to interpet them: https://learn.microsoft.com/en-us/power-query/power-query-online-limits 

A solution would be to go PPU and setup incremental refresh, but I always like to keep my customers on a cheaper license when possible.

Hopefully someone can point me in the right direction for a solution.

[v-karpurapud · Accepted solution]

Hi @BBConsultancy 

Welcome to the Microsoft Fabric Community Forum.

This issue often arises when using Power BI Dataflows (Gen1) with external connectors such as Invantive for Exact Online. While your dataflow usually finishes in about 15 minutes, the error suggests the connection to the data source is ending too soon, likely because of server timeouts or the large amount of data (around 600,000 rows).

Invantive reports that these disconnects are common when Power BI or the user interface cancels data retrieval, possibly after delays or inactivity. To avoid the need for Premium Per User licensing, you can partition your data in Power Query, using filters such as ModifiedDate. By setting StartDate and EndDate parameters, you can break the data into smaller sections, making each partition a separate query in the dataflow, which can later be combined in your semantic model.

It is also helpful to apply filters early in the Invantive query to reduce the amount of data sent to Power BI, improving performance and helping to prevent timeouts. If you need more control over refreshes, consider using Microsoft Fabric’s Gen2 Dataflows, which provide better performance and integration.

For further information on dataflow limitations and troubleshooting, see Microsoft’s documentation on Power Query Online limits and Dataflows Limitations, restrictions and supported connectors and features - Power BI | Microsoft Lea....


Regards,
Karpurapu D,
Microsoft Fabric Community Support Team.