← Back
Self-Study · O'Reilly

Learning Deep Learning

Six parts building a complete understanding of modern deep learning — from perceptrons and backpropagation through CNNs, LSTMs, and the Transformer architecture, applied to translation and image captioning.

Foundations of Deep Learning

PERCEPTRONx₁w1x₂w2x₃w3Σwx+bstepθŷSIGMOID NEURON σ(z)z1.00.5σ(z) = 1/(1+e⁻ᶻ)BACKPROPAGATIONInput xHidden hOutput ŷLoss L∂L/∂wchain rulegradient flow
  • Built intuition for the perceptron — a linear threshold unit that fires when a weighted sum of inputs exceeds a bias, laying the conceptual groundwork for all neural networks
  • Replaced step functions with sigmoid neurons whose smooth, differentiable output enables gradient-based learning: small weight changes produce small, predictable output changes
  • Derived backpropagation from first principles using the chain rule — understanding how gradients flow backward through a network layer by layer to update every weight efficiently
PerceptronSigmoidGradient DescentBackpropagationChain RuleLoss FunctionsLearning Rate