Hi there 👋

Recently, I built my own neural network from scratch in C++, and went through the ordeal of learning how gradient descent and backpropagation work. So, I thought sharing my own explanation could help solidify both my understanding and that of any readers. Here goes! Somewhat unintuitively, backpropagation has 2 steps: forward propagation and backward propagation. We first propagate the input forward through the network to get the activations at each layer. Then, we calculate the error at the output layer and propagate that error “backward” through the layers to calculate gradients for the loss function. I’ll explain what all of this means below. ...