Calculus once there is more than one parameter. Partial derivatives, the Jacobian's shape convention, saddle points, directional derivatives, and the least-squares gradient everything else is built on.
Learning Objectives
→Compute partial derivatives and assemble them into a gradient of the right shape
→Build a Jacobian with outputs down and inputs across, and relate it to dX = dZ @ W.T
→Compute a Hessian, use its symmetry as a check, and say why saddles dominate in high dimensions
→Compute a directional derivative and prove from it that the gradient is the steepest direction
→Derive the least-squares gradient and verify it by shapes alone