Small portion of data imported using data gateway

[Anonymous]

I have a report that uses the Enterprise Data Gateway to pull data (import) from an on-prem SQL server.

Occasionally, the scheduled refresh will only pull minimal records (less than 50) from the sql table, when there are over 5000 records.

The refresh status says "successful".

The gateway status is functional, port tests are all good, other report using both import and directquery are functioning.

I can't find any errors in the gateway logs, but I'm not really sure where to look.

Any tips to find out what is going on?

[v-lionel-msft]

Hi @Anonymous ,

Did you filter the data in "Edit Query"? PBI only refreshes the data you have filtered.

Now, what is the size of your dataset? And what's your license?(Pro or Premium?)

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous · Accepted solution]

It was not filtered in any way.

It turns out the PBI refresh was happening at the EXACT same time as the data refresh in our data warehouse. Therefore it was refreshing before complete and only getting partial records.

[GilbertQ]

hi @Anonymous 

Could you check the query as it appears that it is configured/written to only bring in hte updated data.

[Anonymous]

I have never used incremental refresh, none of my reports use it.

It is happening again today.

In Desktop, I refresh and and get 8437 records. Then I publish to the service and refresh and only get 894 records.

[GilbertQ]

Hi @Anonymous 

Depending if you have set up incremental refresh, if you have it should only get the latest data for the past 1 day.

Otherwise it should get all the data from the SQL Table.

[Anonymous]

I am not using incremental refresh.

The entire table should be imported on each refresh.

[GilbertQ]

Hi @Anonymous 

Sorry I still dont follow what is the issue, you are not getting the 50 rows or the 5000 + 50 rows?

[Anonymous]

I am not getting the 5000 total rows.

[GilbertQ]

Hi @Anonymous 

What is your issue, I could not understand it from your descrption above?

[Anonymous]

The refresh import process is only importing less than 50 records from SQL.

The table has over 5000 records.

There are no errors, and no filters.