Forum Discussion
Spooling Data Gateway
Hi,
We are just testing a new data gateway on an Azure VM, the data volume used in the current test data flow is much bigger than we'd tried previously (50 million records coming from an Azure SQL database). It seems that the whole dataset is spooling before its moved to the Power BI Service,taking a few hours, is this the expected behaviour? I'd previously thought spooling was for a situation where overflow occurred and data wasn't moving fast enough through the gateway, the machine spec is inline with those recommended in the documentation, though there is only one virtual NIC so perhaps throughput may be a bottleneck.
Appreciate any clarification/advice!
Thanks
Hi Anonymous ,
Data is temporarily stored on the gateway machine. This data storage continues until all data is received from the data source. The data is then sent back to the cloud service. This process is called spooling.
And we can't set the spooling location to a custom directory, which is located in the installation directory of the gateway:
C：\ Users \ PBIEgwService \ AppData \ Local \ Microsoft \ In-premises data gateway \ Spooler
Therefore, when installing the gateway,We recommend you use a solid-state drive (SSD) as the spooling storage.
Links of related gateways and spools:
https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-onprem-indepth
Best Regards,
Liu Yang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
2 Replies
Hi Anonymous ,
Data is temporarily stored on the gateway machine. This data storage continues until all data is received from the data source. The data is then sent back to the cloud service. This process is called spooling.
And we can't set the spooling location to a custom directory, which is located in the installation directory of the gateway:
C：\ Users \ PBIEgwService \ AppData \ Local \ Microsoft \ In-premises data gateway \ Spooler
Therefore, when installing the gateway,We recommend you use a solid-state drive (SSD) as the spooling storage.
Links of related gateways and spools:
https://docs.microsoft.com/en-us/data-integration/gateway/service-gateway-onprem-indepth
Best Regards,
Liu Yang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi Liuyang, dose that mean if gateway installed on VM could potentailly have spooling issue?
I've created a ticket but issues still remains, wondering if you could kindly advise?
