Medium

Have a go at it. The editor and the docs panel are open, and your code is saved as you type. Running it needs a free account — you’ll come back to exactly what you wrote.

Cross-Validation

~12 mincode completion

A single train/test split gives you one number, and that number depends on which rows happened to land in the test set. On a small dataset the difference between two splits can be larger than the difference between two models.

k-fold cross-validation splits the data into k parts, then trains k times, each time holding out a different part:

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5)
scores.mean(), scores.std()

Every row is used for training k-1 times and for testing exactly once. You get k scores, and the spread matters as much as the mean: a mean of 0.85 with a standard deviation of 0.02 is a real result, while 0.85 with a standard deviation of 0.15 means you have learned almost nothing about the model.

Two rules:

Pass a pipeline, not pre-scaled data. Otherwise every fold's statistics have leaked into every other fold.

For time series, do not use it. Random folds train on the future to predict the past. Use TimeSeriesSplit.

Your task:

Write cv_mean_score(X, y, folds) that runs cross-validation on a LogisticRegression (with random_state=0) and returns the mean score as a float.

Example Tests

A perfectly separable dataset scores 1.0 across every fold

Input: {"X":[[0],[1],[2],[8],[9],[10]],"y":[0,0,0,1,1,1],"folds":3}

Expected: 1

The same data with two folds still scores 1.0

Input: {"X":[[0],[1],[2],[8],[9],[10]],"y":[0,0,0,1,1,1],"folds":2}

Expected: 1

Python

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

def cv_mean_score(X, y, folds):
    """
    Mean cross-validated accuracy.

    Args:
        X: 2-D feature array
        y: 1-D label array
        folds: how many folds, e.g. 3

    Returns:
        The mean of the per-fold scores, as a float.
    """
    # YOUR CODE HERE
    pass

Run your code to see results

⌘↵ runs against the visible tests

Loading docs…