Power BI is turning 10! Tune in for a special live episode on July 24 with behind-the-scenes stories, product evolution highlights, and a sneak peek at what’s in store for the future.
Save the dateEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Hi All,
I am having an issue using incremental refresh with dataflows, which I am not sure how to resolve.
Current Setup: The dataflow is connected vie "Web" connector and consumes data from an API (so no gateway is needed). the dataflow refreshes fine without setting up the incremental refresh. However, the moment I try to configure the Incremental refresh I get this message:
"Error: Data Source Error : DataSource.Error: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."
I have a DateTime column to use, and I am only looking to store the past 2 years with a daily refresh.
Not sure what I am missing? Any idea/help please?
Thanks
This needs some more details. Does your API support time range filters? Did you apply them in the URL? How?
My source is JIRA and I believe it supports time range filters. The issue is that the dataflow runs with a FULL refresh in less than 2 mins with no issue. Only when I try to configure the Incremental Refresh, it keeps failing and gives the error message.
Configuration: Store rows for the last 2 years and refresh for 1 day.
The issue is that the dataflow runs with a FULL refresh in less than 2 mins with no issue.
I fail to see the issue here 🙂 What problem are you trying to solve?
What I meant here is that the dataflow runs with no issues IF I run a full refresh. But once the full refresh is complete and I try to configure the incremental refresh, it keeps failing.
In summary, if a dataflow runs with a full refresh in less than 2 mins, what can cause the incremental refresh to fail and give the referenced error?
PS: The full refresh step is to troubleshoot if the dataflow has any errors or timeout issues which may affect the Incremental refresh. That's why I mentioned
The issue is that the dataflow runs with a FULL refresh in less than 2 mins with no issue.
I hope that helps.
If your full refresh finishes within two minutes then you don't need to setup the incremental refresh. Performance has to be two orders of magnitude worse to consider incremental refresh.
I appreciate your view but avoiding the incremental refresh issue won't solve my problem 🙂
So far from testing, this is not due to DB size or timeout. My goal is to understand the root cause and what went wrong for this to happen. Could it be the JIRA server refusing the connection? But if so, why would it accept the full refresh?
You would have to provide more details on your incremental refresh setup. At a minumum please show a sanitized version of your Power Query code.