spooq.transformer.mapper_transformations.has_value

has_value(source_column=None, name=None, **kwargs: Any) → partial[source]
Returns True if the source_column is

  • not NULL and

  • not “” (empty string)

  • otherwise it returns False

Warning

This means that it will return True for values which would indicate a False value. Like “false” or 0!!!

Parameters

  • source_column (str or Column) – Input column. Can be a name, pyspark column or pyspark function

  • name (str, default -> derived from input column) – Name of the output column. (.alias(name))

Keyword Arguments

  • alt_src_cols (str, default -> no coalescing, only source_column) – Coalesce with source_column and columns from this parameter.

  • cast (T.DataType(), default -> T.BooleanType()) – Applies provided datatype on output column (.cast(cast))

Examples

>>> input_df = spark.createDataFrame(
...     [
...         Row(input_key=False),
...         Row(input_key=None),
...         Row(input_key="some text"),
...         Row(input_key="")
...     ], schema="input_key string"
... )
>>>
>>> input_df.select(spq.has_value("input_key")).show(truncate=False)
+---------+
|true     |
|false    |
|true     |
|false    |
+---------+
>>>
>>> mapping = [
...     ("original_value",      "input_key",  spq.as_is),
...     ("has_value",  "input_key",  spq.has_value)
... ]
>>> output_df = Mapper(mapping).transform(input_df)
>>> output_df.show(truncate=False)
+--------------+---------+
|original_value|has_value|
+--------------+---------+
|false         |true     |
|null          |false    |
|some text     |true     |
|              |false    |
+--------------+---------+
Returns

This method returns a suitable type depending on how it was called. This ensures compability with Spooq’s mapper transformer - with or without explicit parameters - as well as direct calls via select, withColumn, where, …

Return type

partial or Column