Batch Size, Throughput and Latency

~14 mincode completion

Implement batching_tradeoff(t0, c, arrival_rate, batch_sizes) returning an array of shape (len(batch_sizes), 2) whose rows are [throughput, latency].

Where this shows up

~14 min

8 employers weight this skill

4 big tech firms, 1 AI product company, 1 enterprise vendor, 1 defense company, 1 quant fund. Top match scores 93.

Python
import numpy as np


def batching_tradeoff(t0, c, arrival_rate, batch_sizes):
    """
    Throughput and average per-request latency at several batch sizes.

    Args:
        t0:           fixed cost of one model call, in ms
        c:            extra cost per row in the batch, in ms
        arrival_rate: requests arriving per second
        batch_sizes:  list of batch sizes B

    Returns:
        (len(batch_sizes), 2) array of [throughput in req/s, latency in ms]
    """
    # YOUR CODE HERE
    pass

Run your code to see results

⌘↵ runs against the visible tests

Loading docs…

The AI Mentor needs an account

It reads your code and the failing tests and nudges you toward the fix without handing you the answer. Free accounts get it on every problem you're working on today.