Forum Discussion
Spark SQL Query errors - Case Sensitivity issues
Hi 12angrymentiger
Apologies for the issue you have been facing. At this time, we are reaching out to the internal team to get some help on this. We will update you once we hear back from them.
A simple workaround to fix this issue is to run the below piece of code before executing the main code.spark.conf.set('spark.sql.caseSensitive', False)Hope this helps. Please let me know if you have any further questions. Glad to help.
Hi 12angrymentiger ,
I tried to repro the scenario, but did not get any error. I have attached the screenshot.
Please try to check the Spark version which you ar using.
Make sure you use the Spark 1.2 version.
Hope this helps. Please let me know if you have any further queries.
By the way this is still in My Workspace and under the Fabric Trial, but I did change the workspace to use Runtime 1.2 and I'm still getting the error table or view not found.
Hi 12angrymentiger
Apologies for the issue you have been facing. At this time, we are reaching out to the internal team to get some help on this. We will update you once we hear back from them.
A simple workaround to fix this issue is to run the below piece of code before executing the main code.spark.conf.set('spark.sql.caseSensitive', False)Hope this helps. Please let me know if you have any further questions. Glad to help.
We figured out when you call another notebook and start using spark sql further down in the main notebook the spark sql caseSensitive is True.
We have moved this code to run after the 2nd notebook call to make sure the main notebook spark sql is caseInsensitive
spark.conf.set('spark.sql.caseSensitive', False)