topic Re: Import table incrementally from ODBC on prem in Dataflow

Import table incrementally from ODBC on prem

lench — Tue, 24 Jun 2025 15:27:36 GMT

Hi, i'm trying to import tables from on prem ODBC gateway. I use Dataflow gen2. I have a table with 32 million records.

I've imported it once (took 4 hours).

Now i would like to schedule an hourly incremental update. So what i do is import the table as a query and then i apply a filter on the 'updateDate' col (holding 'Date' format data).

So i've applied a test filter to return only records from the last 2 dates. The problem is that when i run the update, it takes a couple of hours.

Is there a smarter / faster way to do it?

Thank you!!

Re: Import table incrementally from ODBC on prem

lbendlin — Tue, 24 Jun 2025 22:28:41 GMT

The minimum granularity for incremental refresh is "day", as in "daily partitions".

And since "Today" means something different to the Power BI service than what it means to you (Remember the Power BI service runs on UTC) it is very common that two partitions will be refreshed rather than one.

Consider using advanced incremental refresh where the "Today" partition is kept in Direct Query mode.

Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn

Re: Import table incrementally from ODBC on prem

v-saisrao-msft — Thu, 26 Jun 2025 04:21:17 GMT

Hi lench,

I hope you had a chance to review the solution shared by lbendlin. If it addressed your question, please consider accepting it as the solution — it helps others find answers more quickly.
If you're still facing the issue, feel free to reply, and we’ll be happy to assist further.

Thank you.

Re: Import table incrementally from ODBC on prem

v-saisrao-msft — Tue, 01 Jul 2025 04:12:45 GMT

Hi lench,

checking in to see if your issue has been resolved. If the response provided was helpful, kindly mark it as the solution so that others with the same issue can benefit.
Please let us know if you still need assistance.

Thank you.

Re: Import table incrementally from ODBC on prem

v-saisrao-msft — Fri, 04 Jul 2025 04:03:48 GMT

Hi lench,

We haven’t heard back from you in a while. If your issue has been resolved, Mark the relevant response as the solution to help fellow community members.
If you still need support, let us know if you still need assistance.

Thank you.