Forum Discussion

motoray's avatar
motoray
Icon for Advocate V rankAdvocate V
2 years ago

Data Refresh Errors after Enterprise Gateway upgrade

After upgrading to the October 2024 gateway, I'm getting refresh errors randomly for scheduled Power BI refreshes. The error is: "DM_GWPipeline_Gateway_DataSourceAccessError","pbi.error":{"code":"DM_...
dataflow
JohnJ2's avatar
JohnJ2
Regular Visitor
1 year ago

Same error here too of: 

Data source error:{"error":{"code":"DM_GWPipeline_Gateway_DataSourceAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_DataSourceAccessError","parameters":{},"details":[],"exceptionCulprit":1}}} Table: Specialists

where "Table" is always listed as something different.  Have done everything you listed as well:

configured CPUUtilizationPercentageThreshold to 70 from 0
Enabled query caching on jobs using import mode
Enabled Large Semantic Model Storage Format
Enabled Query Scale out

Error is completely random, no consistent time or reason.  Semantic model is nowhere near exceeding the 25GB size limit.  MS just keeps requestintg logs, logs, logs but because the refresh failure throws randomly, we are having a hard time capturing it in logs and when we do submit, no reason is provided from Microsoft.  Frustrating!
  • motoray's avatar
    motoray
    Icon for Advocate V rankAdvocate V
    1 year ago

    I hear you on with the frustration. I did finally get to the next level of tech support...and they wanted more logs. I've found two things that brought some relief--but we still see daily refresh failures.

     

    1. For some reason we get Personal mode connections occasionally in our gateway connections list. We do not use Personal gateways. The Personal mode connections don't connect to anything and can't be changed. If we are careful to check and remove these connections, we get fewer failures.

    2. I found that one of the gateway cluster servers was very low on hard drive space. I had our server folks increase HDD space on both servers in the cluster and bring them up to a higher spec in line with what Microsoft recommends for gateway cluster servers. There was a noticeable difference in # of refresh failures.

     

    Maybe take a look at those and see whether you can get any relief that way.

    • lbendlin's avatar
      lbendlin
      Icon for Super User rankSuper User
      1 year ago

      You also want to consider putting the heavy load items (spooler and logging) onto a different spindle.

      • JohnJ2's avatar
        JohnJ2
        Regular Visitor
        1 year ago

        Yup redirected those to the unlimited E: drive long ago

         

  • kymramosrpo's avatar
    kymramosrpo
    Icon for Advocate V rankAdvocate V
    1 year ago

    have been experiencing the exact same issue here for weeks. Error logs are of no help at all. Have you found a solution, or even a way to diagnose the problem?

    • motoray's avatar
      motoray
      Icon for Advocate V rankAdvocate V
      1 year ago

      We were given a solution that's worked well enough for us with the promise of Microsoft doing some kind of more permanent fix in a future gateway.

       

      There's a configuration file--Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config. On our servers, it's here "C:\Program Files\On-premises data gateway." 

       

      Open the config file in a text editor and scroll down to almost the bottom. Find the "NextResultSkipped" setting and change the value from False to True. After that, restart your gateway and see if the failures stop. They did for us. I never got a satisfactory answer about why they stopped.

      • kymramosrpo's avatar
        kymramosrpo
        Icon for Advocate V rankAdvocate V
        1 year ago

        Thanks for sharing! I've set it up just now. Hope to see some improvements soon. Will share the results here!