PySpark DataFrame Fill Null Values with fillna or na.fill Functions

Code description

In PySpark, DataFrame.fillna, DataFrame.na.fill and DataFrameNaFunctions.fill are alias of each other. We can use them to fill null values with a constant value. For example, replace all null integer columns with value 0, etc. 

Output:

    +--------------+-------+--------+
    |       str_col|int_col|bool_col|
    +--------------+-------+--------+
    |Hello Kontext!|    100|    true|
    |Hello Context!|      0|    null|
    +--------------+-------+--------+
    
    +--------------+-------+--------+
    |       str_col|int_col|bool_col|
    +--------------+-------+--------+
    |Hello Kontext!|    100|    true|
    |Hello Context!|   null|   false|
    +--------------+-------+--------+
    
    +--------------+-------+--------+
    |       str_col|int_col|bool_col|
    +--------------+-------+--------+
    |Hello Kontext!|    100|    true|
    |Hello Context!|      0|   false|
    +--------------+-------+--------+  
    

Code snippet

    from pyspark.sql import SparkSession
    
    app_name = "PySpark fillna"
    master = "local"
    
    spark = SparkSession.builder         .appName(app_name)         .master(master)         .getOrCreate()
    
    spark.sparkContext.setLogLevel("WARN")
    
    # Create a DataFrame
    df = spark.createDataFrame(
        [['Hello Kontext!', 100, True], ['Hello Context!', None, None]],
        ['str_col', 'int_col', 'bool_col'])
    
    # Only fill integer columns
    df.fillna(0).show()
    
    # Only fill boolean columns
    df.fillna(False).show()
    
    # Fill both at the same time
    df.fillna({'int_col': 0, 'bool_col': False}).show()