I am learning apache-spark. And what seems confusing is,
this command works
results = spark.sql("select * from df_sql_view where name LIKE '%i%'")
but this throws an error
results = spark.sql("select * from df_sql_view where name ILIKE '%i%'")
ParseException:
mismatched input 'ILIKE' expecting {<EOF>, ';'}(line 1, pos 37)
== SQL ==
select * from df_sql_view where name ILIKE '%i%'
-------------------------------------^^^
Could someone please help me understand why and how to resolve it. Thank you.
CodePudding user response:
Spark SQL only support like
and rlike
. So you'd want to use rlike
with proper regex to replace ilike