Easy

Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.

Missing Values

~11 mincode completion

Missing data is represented by NaN ("not a number"). It has one property that causes most of the confusion around it: NaN is not equal to anything, including itself.

np.nan == np.nan       # False!
df["x"] == np.nan      # all False, never use this
df["x"].isna()         # the correct way to find them

Finding and handling them:

df.isna().sum()            # how many missing per column. Run this on every new dataset.
df.dropna()                # drop any row with any missing value
df["x"].fillna(0)          # replace with a constant
df["x"].fillna(df["x"].mean())    # replace with the column mean

Which to choose matters. dropna() on a wide table can delete most of your rows, because a row only needs one gap to go. Filling with the mean keeps the row but pulls the column's spread toward the centre and quietly invents data.

And the one that will bite you later: if you fill with a mean computed over the whole dataset, information from your test set has leaked into your training set. The mean must come from training data only. You will meet this again as leakage.

Your task:

Write fill_missing_with_mean(df, column) that replaces the missing values in one column with that column's mean, and returns the DataFrame.

Example Tests

The gap is filled with the mean of the values that were present

Input: {"df":{"x":[1,null,3]},"column":"x"}

Expected: [1,2,3]

A column with no gaps is left exactly as it was

Input: {"df":{"x":[4,6]},"column":"x"}

Expected: [4,6]

No rows are dropped, the row count is unchanged

Input: {"df":{"x":[1,null,null,5]},"column":"x"}

Expected: 4

Python

import pandas as pd

def fill_missing_with_mean(df, column):
    """
    Fill NaNs in one column with that column's mean.

    Args:
        df: a DataFrame
        column: the name of the column to fill

    Returns:
        The DataFrame with that column's gaps filled.
    """
    # YOUR CODE HERE
    pass

def _values(df, column):
    import pandas as pd
    return [float(v) for v in fill_missing_with_mean(pd.DataFrame(df), column)[column]]


def _row_count(df, column):
    import pandas as pd
    return int(len(fill_missing_with_mean(pd.DataFrame(df), column)))

Run your code to see results

⌘↵ runs against the visible tests

Loading docs…