Forum Discussion
jeffshieldsdev
Solution Sage
Solution SageCOUNT is 0 when nullable column in where clause?
I'm having trouble understanding this--if a column contains any nulls, do I have to use coalesce to perform a string comparison on the column? select count(*)
from my_table
-- 292575
select count(...
There is a NULL-safe equality operator in SparkSQL <=>
This properly handles NULLs and equality and can replace the need for coalesces.
https://spark.apache.org/docs/3.5.2/sql-ref-null-semantics.htmlA solution is not present in this thread, despite someone accepting a reply as a solution.
spencer_sa
Impactful Individual
Impactful IndividualThere is a NULL-safe equality operator in SparkSQL <=>
This properly handles NULLs and equality and can replace the need for coalesces.
https://spark.apache.org/docs/3.5.2/sql-ref-null-semantics.html
jeffshieldsdevSolution Sage
Solution SageThanks for the reply. That is producing the same as COALESCE()
I guess I'm surprised Spark handles nulls this way.