topic Re: Unable to load DLL 'ParquetSharpNative': in Dataflow

Unable to load DLL 'ParquetSharpNative':

frankmroberts — Fri, 02 Feb 2024 13:52:34 GMT

I am testing out fabric for my organization and one of our key things to figure out is how to attached to a vendors MySQL database on AWS and bring it back to a LakeHouse. I have created a Gen2 Dataflow that uses our on prem Data Gateway. I then provide it the login info for the MySQL database on AWS and I can succefully see all the table in the database, so that part seems succeful. However when I attempt to bring it back to the LakeHouse in Fabric it fails. Error messages can be seen below.

There was a problem refreshing the dataflow. Please review the error message(s) below, fix the problem, and try again. (Request ID: ee843b8f-21f2-4a4b-a0ee-9ad495a060b6)
flightspan_maf_test aircraft: Error Code: 999999, Error Details: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Unable to load DLL 'ParquetSharpNative': A dynamic link library (DLL) initialization routine failed. (Exception from HRESULT: 0x8007045A) Details: GatewayObjectId: 6b63dd19-d7ee-4ee0-8a3a-0bb4e48f5b66 (Request ID: ee843b8f-21f2-4a4b-a0ee-9ad495a060b6).

After the process runs, I do seem to get a empty parquet file and a _delta_log file in the Tables section of the LakeHouse. All the files have a size of 0.

To rectify the situation we have attempted several things:

1) Per a form post we have upgraded to the latest version of the Data Gateway - no luck

2) I have stripped the fields moved over to the DataLake to just two fields - no luck

3) Turned off Enable Staging on the table

Any ideas?

Thanks for your Help

Frank

Re: Unable to load DLL 'ParquetSharpNative':

Anonymous — Mon, 05 Feb 2024 07:07:43 GMT

Hi frankmroberts ,

Thanks for using Fabric Community.

Apologies for the issue you have been facing. I would like to check are you still facing this issue?
It's difficult to tell what could be the reason for this performance. I would request you to wait for sometime and try again.

If the issue still persists, please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

Re: Unable to load DLL 'ParquetSharpNative':

frankmroberts — Mon, 05 Feb 2024 14:39:27 GMT

Anonymous Thanks for looking at my thread. I have re-created the dataflow this morning and it appears to have the same error. I have placed a support ticket on this and the case number is 2402020010002491

Let me know if there are any other details I can provide. This is a pretty important trial for our ogranization and if Fabric can't create a dataflow for this database I will need to pass on using fabric.

Thanks for your help

Re: Unable to load DLL 'ParquetSharpNative':

miguel — Mon, 05 Feb 2024 16:17:26 GMT

This could potentially be related to the machine where the gateway is installed.

What happens if you use a different machine to run / host the gateway?

Re: Unable to load DLL 'ParquetSharpNative':

frankmroberts — Mon, 05 Feb 2024 16:50:36 GMT

miguel I was also thining that. I have started down the road to install a data gateway (personal mode). But based on the feedback on my ticket with MS, they didn't think that was the issue. I did see in another post where someone downgraded there gatway to the Sept 2023 version and this fixed their issue. I am waiting on what I here from MS, but if I don't here back by the end of the day I might give the data gateway (personal mode) a shot. I am new to the use the of the data gateway, so it is unclear to me how it is configured, if anyone has a link to that, that would be amazing. I do have it installed on my machine, but I assume there is some configuration that must occur to get it working correctly.

Re: Unable to load DLL 'ParquetSharpNative':

miguel — Mon, 05 Feb 2024 17:22:10 GMT

Definitely give it a try. Try installing the gateway on a different machine and use that new gateway installation for your dataflow. Then see if the issue persists or not.

Do continue talking with the support team and if you see that things work after the new installation, then report those back to the support team.

Re: Unable to load DLL 'ParquetSharpNative':

frankmroberts — Mon, 19 Feb 2024 06:29:44 GMT

Anonymous

its been over a week and the support analyst has not responded to my emails and teams posts. Is there some other way to communicate with them that I am not aware of. The problem still exists, however I was successful at coming up with a workaround using a Spak Notebook. But it would still be nice to get to the bottom of this topic. Thanks for your help!

Re: Unable to load DLL 'ParquetSharpNative':

JayJay11 — Tue, 20 Feb 2024 09:38:04 GMT

Hello all,

also we are having a similar issue using an On-Premise Gateway (same error message, but in Gateway logs). After downgrading again to September 2023 version, everything works again.

miguel Anonymous There seems to be an issue with Gateway versions after September 2023. Do you have an ETA on a bugfix? For the moment, we continue using the older version of the Gateway.

We have an open ticket with the tracking ID #2402090050003022.

Re: Unable to load DLL 'ParquetSharpNative':

cieplag — Tue, 05 Mar 2024 09:53:27 GMT

Hi!

Do you have any update on this topic through your ticket?

Re: Unable to load DLL 'ParquetSharpNative':

JayJay11 — Tue, 05 Mar 2024 10:12:00 GMT

Hey cieplag,

the ticket has been closed because we could "solve" the issue with the Septmeber 2023 version of the gateway.

However, I am also interested what is the status on this issue miguel Anonymous

Re: Unable to load DLL 'ParquetSharpNative':

miguel — Tue, 05 Mar 2024 11:52:29 GMT

The issue that you're having could be different to the one initially reported. I'd definitely recommend reopening that ticket so an engineer can get to the bottom of it.

These sorts of issues are typically specific to an environment where the gateway is installed and not something specific to the gateway, so an investigation with the support team is recommended .

Re: Unable to load DLL 'ParquetSharpNative':

frankmroberts — Tue, 05 Mar 2024 16:18:01 GMT

As of last week I was told Microsoft was still looking into it. Here is what they said on 2/28:

Regarding the issue I checked with my TA and suggested me to escalate the issue to the product team individually for this ticket and following the same I will be providing you with the updates whatever I will get on the same.

Re: Unable to load DLL 'ParquetSharpNative':

JayJay11 — Fri, 08 Mar 2024 09:39:59 GMT

Thanks - updates are very highly appreciated. Unfortunately as of now MS Support is not responding anymore to my e-mails (I agreed with Support to close the ticket but they would follow-up with me on any updates on the issue).

I assume it will be an important indicator as of when we can update the On-Prem Gateway from Sep 2023 to the newest version.

Re: Unable to load DLL 'ParquetSharpNative':

jawa — Wed, 13 Mar 2024 13:15:49 GMT

I was able to resolve

Unable to load DLL 'ParquetSharpNative': A dynamic link library (DLL) initialization routine failed

by downgrading to September 2023 release of data gateway.

I used the download link here https://powerbi.microsoft.com/en-us/blog/on-premises-data-gateway-september-2023-release/

Re: Unable to load DLL 'ParquetSharpNative':

pqian_MSFT — Wed, 13 Mar 2024 18:38:33 GMT

Hey everyone, I believe we have narrowed down the ParquetSharp issue on post-NOV-23 gateways to compilation options against the AVX-2 instruction set, which does not work on older CPUs or hyper-v servers that has compatibility mode turned on.

If you are on older CPUs: wait for an update

If you are on Hyper-v that has compatibility mode on: try turning it off.

Re: Unable to load DLL 'ParquetSharpNative':

frankmroberts — Wed, 13 Mar 2024 19:55:00 GMT

So to be clear, is this an issue with where the gateway is installed? Are you saying turn off Hyper-v compatibility mode off on that machine where the gateway is installed? If so will this then be fixed on the coming version of the gateway? When will that come out?

Thanks for your help on this!

Re: Unable to load DLL 'ParquetSharpNative':

pqian_MSFT — Wed, 13 Mar 2024 19:57:18 GMT

Yes, it's about where the GW is installed. It'd be interesting to confirm how many people on this thread is running inside Hyper-V with compatibility mode on.

It'll be fixed and QFEd. Expect this within 1-2 business weeks.

Re: Unable to load DLL 'ParquetSharpNative':

pqian_MSFT — Sat, 16 Mar 2024 06:28:38 GMT

frankmroberts JayJay11 jawa cieplag The patched gateway version 3000.214.8 is now live. Please download it and install it again.

Re: Unable to load DLL 'ParquetSharpNative':

cieplag — Thu, 21 Mar 2024 13:08:31 GMT

I have verified that everything works now after installing the March version. Thank you!

Re: Unable to load DLL 'ParquetSharpNative':

StuartTurner — Wed, 08 May 2024 16:51:21 GMT

Is there a known issue/ recommendation for VMWare machines ? We have installed the latest Gateway version but are still getting this error