Forum Discussion
Solved
input_file_name() function not working when using custom Environment
EDIT; After a bit more testing I found out that the input_file_name() function is not working when native execution engine is enabled in the Environment attached to the notebook. Any ideas why t...
HI RickPeters,
Perhaps you can take al ooak the following document about the spark native execution engine and its limitations if helps:
Native execution engine for Fabric Spark - Microsoft Fabric | Microsoft Learn
Regards,Xiaoxin Sheng
Thanks for this! I was playing around with some Spark configurations and input_file_name also failed and ruined my Data Pipeline. I was unable to find the exact cause, and this helps a lot.
It's a real shame when stuff like this breaks a fairly stable ETL prod process.