Forum Discussion
Native execution engine enabled by default in fabric spark?
I keep running into tons of spark bugs, like the one described here and more:
The error message we encounter often looks like so:
The Code_Verifier does not match the code_challenge supplied in the authorization request
The docs say this feature is in preview:
Native execution engine for Fabric Spark - Microsoft Fabric | Microsoft Learn
Is this stuff enabled for everyone else? Or is my tenant being treated as a guinea pig?
Given that this is in preview and there are still bugs, why is it taking effect in my environment?
The answer is that Microsoft should not be enabling preview features by default. Fabric is already buggy enough without having to deal with unexpected errors from the preview features as well.
Customers don't want to spend additional time working with Mindtree support, as a results of untested features that are being secretly enabled in our workspaces.
Here is one of the known issues that refers to this error message:
9 Replies
Hi dbeavon3 ,
Thank you for reaching out to us on the Microsoft Fabric Community Forum.The term "public preview" means the feature is available for all users to test, but it may still be under development and subject to changes. The announcement of availability at no additional cost likely refers to the feature being accessible during this preview phase without extra charges.
Given that the feature is in public preview, it's possible that your environment has been included to gather diverse usage data and feedback. If the Native Execution Engine is causing issues in your environment, you have the option to disable it at both the environment and notebook/job levels.
- Environment Level: Navigate to your environment settings in the Microsoft Fabric portal. Under the "Acceleration" tab, you may find a toggle to enable or disable the Native Execution Engine.
- Notebook or Job Level: Within your notebooks or Spark job definitions, you can control this feature by setting the Spark configuration spark.native.enabled to false at the beginning of your script.
Please refer below:
native-execution-engine-overview
Native Execution Engine available at no additional cost!
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.
Regards,
Pallavi.
Hi Anonymous
Thanks for the info. Is this stuff enabled for everyone else? (Can you check on your side?)The purpose is to find out if my tenant being treated as a guinea pig.
My purpose is not to serve Microsoft in the capacity of a beta-tester. If Microsoft wishes to "gather diverse usage data and feedback", then that should be done in an opt-in basis, not opt-out.
NOTE: If something is in preview, then it is probably even less reliable than normal, and even less adequately supported by Mindtree. These are the things that I would NOT want to be enabled by default in our environment.
Please let me know about your environment as well.
Hi dbeavon3 ,
Thank you for the response.
The Native Execution Engine for Fabric Spark is in public preview and has been made widely available, rather than being selectively enabled for specific tenants. While it is not enabled in my environment, it is likely active for many users. If you prefer to disable it, you may do so at the environment level (Fabric portal → Acceleration tab) or at the notebook/job level, as mentioned earlier. Refer the link for more information
Hope this helps.If the response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you for being a valued member in Microsoft Fabric Community Forum!
- Environment Level: Navigate to your environment settings in the Microsoft Fabric portal. Under the "Acceleration" tab, you may find a toggle to enable or disable the Native Execution Engine.
Hi dbeavon3
I wanted to check in on your situation regarding the issue. Have you resolved it? If you have, please consider marking the reply that helped you as Accept as solution or sharing your solution. It would be greatly appreciated by others in the community who may have the same question.
Thank you for being a valued member of the Microsoft Fabric Community Forum!Hi dbeavon3 ,
We wanted to follow up since we haven't heard back from you regarding our last response. We hope your issue has been resolved.If our answer resolved your query, please mark it as "Accept as Solution" and select "Yes" if it was helpful.If you need any further assistance, feel free to reach out.
Thank you for being a valued member of the Microsoft Fabric Community Forum!
