Error with refresh through on-premise data gateway (Gen2 Dataflow) - "ParquetSharpNative"

[JayJay11]

Hi all,

we have now tested Fabric intensively within the last two weeks. I have to say that when Fabric was announced I was quite hyped and still today I see big potential. However, today I ask myself, what the heck is going on with this product. We are encountering so many issues, errors and bugs it is impossible to achieve anything.

• Loading data from cloud data source to Lakehouse -> Timeout after 8h
• Loading data from on-prem data source to Lakehouse -> Error everytime although everything is configured correctly on the Gateway server
• Loading data from a Gen1 dataflow to Lakehouse -> Error, because of memory allocation??
• Issues that suddenly disappear after tried x times (e.g. OAuth 2.0 not working in Dataflow)

For all these issues, you can believe me that I have tried all possible variations to fix it. Also, we are in contact with MFST support since one week, exchanging many e-mails etc.

Maybe someone at Microsoft can give some context what the current situation is. And: Is someone here in this forum who uses one or many components of Fabric in production? What is your experience so far?

For me, it seems Fabric is incredibly unstable and buggy at the moment. Then I ask myself: Why can we already buy a capacity?

Appreciate all kinds of inputs here ...

[JayJay11 · Accepted solution]

Hi @v-nikhilan-msft , you are right, sorry. We have sent / uploaded the logs this morning via the open ticket #2402090050003022.

Further, we have downgraded the Gateway to September 2023 version and the Gen2 Dataflows work again! It seems also we are not the only ones having this issue since last week.

[pqian_MSFT]

Thanks

- Timeout: I guess we've looked at this before right? This is from your Gen 1 vs Gen 2 dataflow comparison, where the Gen 1 DF was issuing Odata requests and getting an average of 0.5sec response time and Gen 2 was getting 3.5secs. This is from the S4 hana endpoint - is there any server logs to check why the response data rate is slower here? Note that both Gen1 and Gen2 refreshes are from the Germany data center.

- Gateway: This one is interesting, ParquetSharp should be bundled with the GW installer. I found the support case and will follow up internally.

- OAuth 2.0: if you reproduce this let me know. Sometimes the "invalid credentials" can be thrown due to firewall rules (some servers will deny your network access with a 401\403 even if you have the right creds)

I think given your scenarios, the best bet forward is to make your OnPrem Data Gateway work. There's perhaps some opportunities to tweak those OData queries to use the S4HANA connector, but just switching over to OPDG should resolve most of your issues. So let's focus on this one and I'll follow up on your support case to see how we can help with the ParquetSharp problem.

[JayJay11]

Yes, I agree that solving the Data Gateway problem would solve my situation. I can work around the timeout problem as this SAP endpoint is anyway responding very slow, maybe using a blob storage solution in Azure with a delta mechanism going forward. The memory allocation problem would also render irrelevant as I would anyway not want to mix Gen1 and Gen2 dataflows.

Thank you for your support. I hope we can fix the OPDG issue next week or so 🙂

[pqian_MSFT]

Apologies for the issues you are running into, let's look into them?

• Loading data from cloud data source to Lakehouse -> Timeout after 8h

-> Please share a request ID from the refresh history screen.

• Loading data from on-prem data source to Lakehouse -> Error everytime although everything is configured correctly on the Gateway server

-> What is the specific error message here? Can you share one of the refresh history screen with details?

• Loading data from a Gen1 dataflow to Lakehouse -> Error, because of memory allocation??

-> I believe this is Dataflow Gen2 Error: Out of memory: realloc of siz... - Microsoft Fabric Community, let's use that post to gather details.

• Issues that suddenly disappear after tried x times (e.g. OAuth 2.0 not working in Dataflow)

-> What was the error message when "OAuth 2.0 not working"? A session ID\request ID would help here.

[JayJay11]

Hi @pqian_MSFT, here some additional info:

• Timeouts : 930430ce-55a9-433f-b8d5-90308af0619a

• Loading from on-prem to Lakehouse : Actually, I am working together with the support on this. It seems there is an issue with the latest Gateway version, the error found in the Gateway logs looks like the following below. Please note, we have checked the ports on the Gateway server - all good as required by the documentation"Microsoft.Mashup.Engine1.Library.Parquet.Schema.ParquetTypeMaps" caused an exception. ---> System.TypeInitializationException: The type initializer for "Ints" caused an exception. ---> System.DllNotFoundException: The DLL "ParquetSharpNative": A DLL initialization routine failed. (Exception from HRESULT: 0x8007045A) cannot be loaded."

• Memory allocation : See separate thread, I posted the request ID there (b55b0d6d-89b8-47d3-b698-636ba5791825)

• OAuth 2.0 : It just simply stated "Invalid credentials", although they were clearly correct. Even tried to use the browser in guest mode to get rid of cache. After re-creating the data source multiple times in the Gateway it suddenly worked again. I cannot reproduce it now

[kayzid]

Hello,

We apologize for the issues you are running into. For the error - "ParquetSharpNative": A DLL initialization routine failed which you are getting when trying to refresh a gen2 dataflow via gateway,. Can you please capture Process Monitor logs on the gateway machine, by following the below instructions

1. Download the Process Monitor tool from - Process Monitor - Sysinternals | Microsoft Learn

2. Unzip the folder and run procmon64.exe

3. Refresh the Gen2 dataflow via gateway. 

4. Save the Procmon logs and share it (check your private message for sharing instructions).

5. Please ensure to provide the start time and end time for the refresh as it will be helpful in correlating it with the process monitor logs.

The logs which process monitor collects may have more information about why the DLL in question is failing to load.

Thanks!

[JayJay11]

Thank you @kayzid  I will do that next week on Monday with the infrastructure engineer. I'll send the logs as soon as I have them.

[v-nikhilan-msft]

Hi @JayJay11 
We haven’t heard from you on the last response and was just checking back to see if you can provide the details as mentioned above.
Thanks.

[JayJay11 · Accepted solution]

Hi @v-nikhilan-msft , you are right, sorry. We have sent / uploaded the logs this morning via the open ticket #2402090050003022.

Further, we have downgraded the Gateway to September 2023 version and the Gen2 Dataflows work again! It seems also we are not the only ones having this issue since last week.

[dazhang]

Hi @JayJay11, the "ParquetSharpNative": A DLL initialization routine failed" issue should be fixed in the March version of the On-Premises Data Gateway that is now available for download. Would you mind giving that a try and let us know if it resolves the issue for you?

[JayJay11 · Accepted solution]

Hi @dazhang we installed the March version and everything is working again. Thank you!

[dazhang]

Great, thanks for letting us know!