Forum Discussion
Native execution engine enabled by default in fabric spark?
The answer is that Microsoft should not be enabling preview features by default. Fabric is already buggy enough without having to deal with unexpected errors from the preview features as well.
Customers don't want to spend additional time working with Mindtree support, as a results of untested features that are being secretly enabled in our workspaces.
Here is one of the known issues that refers to this error message:
Hi dbeavon3 ,
Thank you for reaching out to us on the Microsoft Fabric Community Forum.
The term "public preview" means the feature is available for all users to test, but it may still be under development and subject to changes. The announcement of availability at no additional cost likely refers to the feature being accessible during this preview phase without extra charges.
Given that the feature is in public preview, it's possible that your environment has been included to gather diverse usage data and feedback. If the Native Execution Engine is causing issues in your environment, you have the option to disable it at both the environment and notebook/job levels.
- Environment Level: Navigate to your environment settings in the Microsoft Fabric portal. Under the "Acceleration" tab, you may find a toggle to enable or disable the Native Execution Engine.
- Notebook or Job Level: Within your notebooks or Spark job definitions, you can control this feature by setting the Spark configuration spark.native.enabled to false at the beginning of your script.
Please refer below:
native-execution-engine-overview
Native Execution Engine available at no additional cost!
If this helps, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.
Regards,
Pallavi.
Hi Anonymous
Thanks for the info. Is this stuff enabled for everyone else? (Can you check on your side?)
The purpose is to find out if my tenant being treated as a guinea pig.
My purpose is not to serve Microsoft in the capacity of a beta-tester. If Microsoft wishes to "gather diverse usage data and feedback", then that should be done in an opt-in basis, not opt-out.
NOTE: If something is in preview, then it is probably even less reliable than normal, and even less adequately supported by Mindtree. These are the things that I would NOT want to be enabled by default in our environment.
Please let me know about your environment as well.