Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

Small portion of data imported using data gateway

I have a report that uses the Enterprise Data Gateway to pull data (import) from an on-prem SQL server.

Occasionally, the scheduled refresh will only pull minimal records (less than 50) from the sql table, when there are over 5000 records.

The refresh status says "successful".

The gateway status is functional, port tests are all good, other report using both import and directquery are functioning.

I can't find any errors in the gateway logs, but I'm not really sure where to look.

 

Any tips to find out what is going on?

Need Help
  • Anonymous's avatar
    Anonymous
    5 years ago

    It was not filtered in any way.

     

    It turns out the PBI refresh was happening at the EXACT same time as the data refresh in our data warehouse. Therefore it was refreshing before complete and only getting partial records.

10 Replies

  • GilbertQ's avatar
    GilbertQ
    Icon for Super User rankSuper User
    5 years ago

    Hi Anonymous 

     

    What is your issue, I could not understand it from your descrption above?

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      The refresh import process is only importing less than 50 records from SQL.

      The table has over 5000 records.

      There are no errors, and no filters.

  • GilbertQ's avatar
    GilbertQ
    Icon for Super User rankSuper User
    5 years ago

    Hi Anonymous 

     

    Sorry I still dont follow what is the issue, you are not getting the 50 rows or the 5000 + 50 rows?

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      I am not getting the 5000 total rows.

  • GilbertQ's avatar
    GilbertQ
    Icon for Super User rankSuper User
    5 years ago

    Hi Anonymous 

     

    Depending if you have set up incremental refresh, if you have it should only get the latest data for the past 1 day.

     

    Otherwise it should get all the data from the SQL Table.

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      I am not using incremental refresh.

      The entire table should be imported on each refresh.

  • GilbertQ's avatar
    GilbertQ
    Icon for Super User rankSuper User
    5 years ago

    hi Anonymous 

     

    Could you check the query as it appears that it is configured/written to only bring in hte updated data.

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      I have never used incremental refresh, none of my reports use it.

      It is happening again today.

      In Desktop, I refresh and and get 8437 records. Then I publish to the service and refresh and only get 894 records.

  • v-lionel-msft's avatar
    v-lionel-msft
    Icon for Community Support rankCommunity Support
    5 years ago

    Hi Anonymous ,

     

    Did you filter the data in "Edit Query"? PBI only refreshes the data you have filtered.

    Now, what is the size of your dataset? And what's your license?(Pro or Premium?)

     

    Best regards,
    Lionel Chen

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

     

     

     

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      It was not filtered in any way.

       

      It turns out the PBI refresh was happening at the EXACT same time as the data refresh in our data warehouse. Therefore it was refreshing before complete and only getting partial records.