Byte-Pair Encoding Merge Step
~20 mincode completion
Implement bpe_merge_step(tokens) returning the new token list after one merge. A list with fewer than two tokens has no pairs, so return it unchanged.
Where this shows up
~20 min
••••••••
4 employers weight this skill
2 frontier labs, 1 AI product company, 1 quant fund. Top match scores 77.
Python
def bpe_merge_step(tokens):
"""
One byte-pair encoding merge.
Args:
tokens: list of strings
Returns:
list of strings with every occurrence of the most frequent
adjacent pair merged (ties broken by the smallest pair)
"""
# YOUR CODE HERE
pass