Causal Attention Mask

Causal Attention Mask

In autoregressive decoding, token $i$ is only allowed to attend to tokens $j\le i$.

If future tokens are visible during training, the model can "cheat".

A causal mask $M\in {\mathbb{R}}^{L\times L}$ is defined as:

This is a lower-triangular matrix. In full attention implementations, entries with 0 are usually turned into $-\infty$ before softmax.

Your task: Implement causal_mask(seq_len) to return a float NumPy array of shape (seq_len, seq_len).

Implementation requirements:

1. Include the diagonal (self-attention is allowed).

2. Return numeric values 1.0 and 0.0.

3. Use a vectorized NumPy approach.