Forum Discussion
Looking for comprehensive documentation about the Data Gateway configuration and traces available
Hi phnslzde ,
Thank you for your detailed follow-up. I recognize the difficulties that arise when the FabricPipelineWorker process fails without clear logs, which can complicate troubleshooting. At present, there is no comprehensive public documentation covering all gateway worker behaviors, but here are several key points:
- Trace configuration: You are monitoring the appropriate gateway logs at C:\Windows\ServiceProfiles\PBIEgwService\AppData\Local\Microsoft\On-premises data gateway. Traces for FabricPipelineWorker.exe are found in IntegrationRuntimeErrors<xxxx>.log and related DataMovement logs. Please note that logging verbosity is controlled globally via the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file; setting this to Verbose will provide more detailed entries, though it will increase log size.
- Real-time log monitoring: Notepad++ with “file change alert” remains the most effective tool for viewing logs in real time. Logs are written asynchronously, so some delay is expected, and immediate line-by-line flushing is not available.
- Worker crash analysis: If FabricPipelineWorker.exe crashes, reviewing the Windows Event Viewer (Application and System logs) can help identify issues such as .NET runtime or memory problems. It is advisable to review these logs alongside gateway logs.
- Resources and connections: Certain drivers or connectors, including Oracle, SAP, and ODBC, may consume more memory than expected, even during basic connection tests, which can result in cancellations without data transfer.
Given that you are using the latest September 2025 build and have updated worker memory settings, I suggest enabling Verbose tracing, reproducing the issue, and collecting both gateway and Event Viewer logs. If the problem continues, submitting a support ticket with these details will help engineering determine whether the cause is related to a connector or resource allocation.
Thank you,Tejaswi.
Hi phnslzde ,
Thank you for bringing this up. The error message:
“Not enough resources to start a new worker. Current workers count: 1”
usually means the gateway host doesn't have enough memory to start more workers. Here are two main steps you can take:
Modify worker memory settings
Go to: C:\Program Files\On-premises data gateway\Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config
Find the setting: OnPremRuntimeDefaultWorkerMaxWorkingSetInMB
Lower this value (for example, to 200–300 MB). If it’s too high, even one worker might use all the memory, which stops new workers from starting.
Upgrade host resources
If your server or VM has limited RAM or CPU (like 4 GB RAM), consider increasing to at least 8 GB RAM and more vCPUs. Each worker needs memory and CPU, so more resources let more workers run at once.
Other tips:
Make sure your gateway is updated to the latest version. .If you run multiple jobs at once, try lowering pipeline concurrency. Restart the gateway service if you think there are stuck worker processes.
Hello, Thank you! I run it with 16 GB and currently 4 GB of RAM free. I changd the parameter to no success. The process "FabricPipelineWorker.exe" Is started when I test a connection. Then it just dies without further details observable for me in the logs. on Fabric I just get the generic error message, sometimes I can find some more. Can I influence trace behaviour of this process? I checked for a .config file related to it but I couldn't find any. That's why I asked for more documentation or comprehensive literature. It is not the only error I have, in relation with another connection type I also observe unclear behaviour. And again, I'm just checking the connection , so no major data transfer , etc is happening.
The gateway is the latest version, September 2025 release. Cheers.Hi phnslzde ,
Thank you for your detailed follow-up. I recognize the difficulties that arise when the FabricPipelineWorker process fails without clear logs, which can complicate troubleshooting. At present, there is no comprehensive public documentation covering all gateway worker behaviors, but here are several key points:
- Trace configuration: You are monitoring the appropriate gateway logs at C:\Windows\ServiceProfiles\PBIEgwService\AppData\Local\Microsoft\On-premises data gateway. Traces for FabricPipelineWorker.exe are found in IntegrationRuntimeErrors<xxxx>.log and related DataMovement logs. Please note that logging verbosity is controlled globally via the Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.dll.config file; setting this to Verbose will provide more detailed entries, though it will increase log size.
- Real-time log monitoring: Notepad++ with “file change alert” remains the most effective tool for viewing logs in real time. Logs are written asynchronously, so some delay is expected, and immediate line-by-line flushing is not available.
- Worker crash analysis: If FabricPipelineWorker.exe crashes, reviewing the Windows Event Viewer (Application and System logs) can help identify issues such as .NET runtime or memory problems. It is advisable to review these logs alongside gateway logs.
- Resources and connections: Certain drivers or connectors, including Oracle, SAP, and ODBC, may consume more memory than expected, even during basic connection tests, which can result in cancellations without data transfer.
Given that you are using the latest September 2025 build and have updated worker memory settings, I suggest enabling Verbose tracing, reproducing the issue, and collecting both gateway and Event Viewer logs. If the problem continues, submitting a support ticket with these details will help engineering determine whether the cause is related to a connector or resource allocation.
Thank you,Tejaswi.
