LLM Judge Position Consistency

~12 mincode completion

Implement position_consistency(v_ab, v_ba) returning the array [consistency, first_rate].

Where this shows up

~12 min

8 employers weight this skill

3 AI product companies, 2 frontier labs, 1 health and bio company, 1 big tech firm, 1 autonomy company. Top match scores 93.

Python
import numpy as np


def position_consistency(v_ab, v_ba):
    """
    Position-bias check for a pairwise LLM judge.

    Args:
        v_ab: positions picked with A shown first, shape (n,), values 0 or 1
        v_ba: positions picked with B shown first, shape (n,), values 0 or 1

    Returns:
        array [consistency, first_rate]
    """
    # YOUR CODE HERE
    pass
Loading docs…

The AI Mentor needs an account

It reads your code and the failing tests and nudges you toward the fix without handing you the answer. Free accounts get it on every problem you're working on today.