Hard

RNN Forward Step

Hard

~20 min

code completion

RNN: Single Time Step

A Recurrent Neural Network (RNN) processes sequences by maintaining a hidden state that is updated at each time step:

h_{t} = tanh (W_{h} h_{t - 1} + W_{x} x_{t} + b)

where:

h_{t - 1}

: previous hidden state, shape (hidden_size,)

x_{t}

: current input, shape (input_size,)

W_{h}

: hidden-to-hidden weight matrix, shape (hidden_size, hidden_size)

W_{x}

: input-to-hidden weight matrix, shape (hidden_size, input_size)

b

: bias, shape (hidden_size,)

tanh

: applied element-wise, squashes to (-1, 1)

Your task:

Implement rnn_step(h_prev, x, Wh, Wx, b) that returns the next hidden state $h_{t}$ .

Example Tests

Zero inputs and prev state: output is tanh(b)

Input: {"b":[0,0],"x":[0,0],"Wh":[[1,0],[0,1]],"Wx":[[1,0],[0,1]],"h_prev":[0,0]}

Expected: [0,0]

Identity Wh, zero Wx and b: output = tanh(h_prev)

Input: {"b":[0,0],"x":[0,0],"Wh":[[1,0],[0,1]],"Wx":[[0,0],[0,0]],"h_prev":[1,0]}

Expected: [0.76159,0]

tanh squashes large values to ±1

Input: {"b":[0,0],"x":[1,0],"Wh":[[0,0],[0,0]],"Wx":[[100,0],[0,1]],"h_prev":[0,0]}

Expected: [1,0]

You can read the full problem statement above. Create a free account to run code in the browser, submit solutions, and track your progress.