Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Anonymous
Not applicable

Small portion of data imported using data gateway

I have a report that uses the Enterprise Data Gateway to pull data (import) from an on-prem SQL server.

Occasionally, the scheduled refresh will only pull minimal records (less than 50) from the sql table, when there are over 5000 records.

The refresh status says "successful".

The gateway status is functional, port tests are all good, other report using both import and directquery are functioning.

I can't find any errors in the gateway logs, but I'm not really sure where to look.

 

Any tips to find out what is going on?

1 ACCEPTED SOLUTION
Anonymous
Not applicable

It was not filtered in any way.

 

It turns out the PBI refresh was happening at the EXACT same time as the data refresh in our data warehouse. Therefore it was refreshing before complete and only getting partial records.

View solution in original post

10 REPLIES 10
v-lionel-msft
Community Support
Community Support

Hi @Anonymous ,

 

Did you filter the data in "Edit Query"? PBI only refreshes the data you have filtered.

Now, what is the size of your dataset? And what's your license?(Pro or Premium?)

 

Best regards,
Lionel Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

 

 

Anonymous
Not applicable

It was not filtered in any way.

 

It turns out the PBI refresh was happening at the EXACT same time as the data refresh in our data warehouse. Therefore it was refreshing before complete and only getting partial records.

GilbertQ
Super User
Super User

hi @Anonymous 

 

Could you check the query as it appears that it is configured/written to only bring in hte updated data.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

I have never used incremental refresh, none of my reports use it.

It is happening again today.

In Desktop, I refresh and and get 8437 records. Then I publish to the service and refresh and only get 894 records.

GilbertQ
Super User
Super User

Hi @Anonymous 

 

Depending if you have set up incremental refresh, if you have it should only get the latest data for the past 1 day.

 

Otherwise it should get all the data from the SQL Table.





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

I am not using incremental refresh.

The entire table should be imported on each refresh.

GilbertQ
Super User
Super User

Hi @Anonymous 

 

Sorry I still dont follow what is the issue, you are not getting the 50 rows or the 5000 + 50 rows?





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

I am not getting the 5000 total rows.

GilbertQ
Super User
Super User

Hi @Anonymous 

 

What is your issue, I could not understand it from your descrption above?





Did I answer your question? Mark my post as a solution!

Proud to be a Super User!







Power BI Blog

Anonymous
Not applicable

The refresh import process is only importing less than 50 records from SQL.

The table has over 5000 records.

There are no errors, and no filters.

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.