Learning Rate Scaling and Training Cost
~8 mincode completion
Implement training_budget(base_lr, batch_size, epochs, n_examples, throughput, num_gpus, price_per_gpu_hour) returning the array [lr, gpu_hours, cost].
Where this shows up
~8 min
••••••••••••••••
8 employers weight this skill
4 big tech firms, 2 quant funds, 1 frontier lab, 1 autonomy company. Top match scores 93.
Python
import numpy as np
def training_budget(base_lr, batch_size, epochs, n_examples, throughput, num_gpus, price_per_gpu_hour):
"""
Scaled learning rate and the cost of a training run.
Args:
base_lr: learning rate that works at a batch of 256
batch_size: batch size B
epochs: passes over the data E
n_examples: training examples N
throughput: examples per second r, across all GPUs
num_gpus: GPUs G billed for the whole run
price_per_gpu_hour: dollars per GPU hour
Returns:
array [lr, gpu_hours, cost]
"""
# YOUR CODE HERE
pass