How do you type hint that a function will return a PySpark Column type?
import pyspark.sql.functions as F
def get_some_filter_expression(col_string_name) -> TODO:
return F.col(col_string_name)
I included a dummy example above, and I am trying to figure out what TODO should be.
EDIT: I guess the reason I had this question was because I was assuming that PySpark "column types" under spark.sql.types
could be used for type-hinting. This is not the case. In case it is a common confusion, I will leave this question as is. See the answer of LiamFiddler below.
CodePudding user response:
You can do the following:
from pyspark.sql import Column, Functions as F
def get_some_filter_expression(col_string_name) -> Column:
return F.col(col_string_name)