Derivatives as rates, met numerically first. Central differences, the chain rule, the sigmoid derivative, gradient checking, and what curvature says about your learning rate.
Learning Objectives
→Estimate a derivative with a central difference and say why it beats a forward one
→Apply the chain rule as outer-times-inner, the operation backpropagation repeats
→Derive the sigmoid derivative and explain the vanishing gradient from its maximum of 0.25
→Check an analytic gradient numerically, component by component
→Compute a second derivative and relate curvature to the largest stable learning rate