Forum Discussion
Error with refresh through on-premise data gateway (Gen2 Dataflow) - "ParquetSharpNative"
Hi Anonymous , you are right, sorry. We have sent / uploaded the logs this morning via the open ticket #2402090050003022.
Further, we have downgraded the Gateway to September 2023 version and the Gen2 Dataflows work again! It seems also we are not the only ones having this issue since last week.
Apologies for the issues you are running into, let's look into them?
- Loading data from cloud data source to Lakehouse -> Timeout after 8h
-> Please share a request ID from the refresh history screen.
- Loading data from on-prem data source to Lakehouse -> Error everytime although everything is configured correctly on the Gateway server
-> What is the specific error message here? Can you share one of the refresh history screen with details?
- Loading data from a Gen1 dataflow to Lakehouse -> Error, because of memory allocation??
-> I believe this is Dataflow Gen2 Error: Out of memory: realloc of siz... - Microsoft Fabric Community, let's use that post to gather details.
- Issues that suddenly disappear after tried x times (e.g. OAuth 2.0 not working in Dataflow)
-> What was the error message when "OAuth 2.0 not working"? A session ID\request ID would help here.
Hi pqian_MSFT, here some additional info:
- Timeouts : 930430ce-55a9-433f-b8d5-90308af0619a
- Loading from on-prem to Lakehouse : Actually, I am working together with the support on this. It seems there is an issue with the latest Gateway version, the error found in the Gateway logs looks like the following below. Please note, we have checked the ports on the Gateway server - all good as required by the documentation"Microsoft.Mashup.Engine1.Library.Parquet.Schema.ParquetTypeMaps" caused an exception. ---> System.TypeInitializationException: The type initializer for "Ints" caused an exception. ---> System.DllNotFoundException: The DLL "ParquetSharpNative": A DLL initialization routine failed. (Exception from HRESULT: 0x8007045A) cannot be loaded."
- Memory allocation : See separate thread, I posted the request ID there (b55b0d6d-89b8-47d3-b698-636ba5791825)
- OAuth 2.0 : It just simply stated "Invalid credentials", although they were clearly correct. Even tried to use the browser in guest mode to get rid of cache. After re-creating the data source multiple times in the Gateway it suddenly worked again. I cannot reproduce it now
Hello,
We apologize for the issues you are running into. For the error - "ParquetSharpNative": A DLL initialization routine failed which you are getting when trying to refresh a gen2 dataflow via gateway,. Can you please capture Process Monitor logs on the gateway machine, by following the below instructions
1. Download the Process Monitor tool from - Process Monitor - Sysinternals | Microsoft Learn
2. Unzip the folder and run procmon64.exe
3. Refresh the Gen2 dataflow via gateway.
4. Save the Procmon logs and share it (check your private message for sharing instructions).
5. Please ensure to provide the start time and end time for the refresh as it will be helpful in correlating it with the process monitor logs.
The logs which process monitor collects may have more information about why the DLL in question is failing to load.
Thanks!
