haotianblog
Deep Learning Tutorials
A Deep Learning and CNN topic hub connecting neural network basics, CIFAR-10 Tiny CNN, convolution, pooling, softmax, and backpropagation.

Deep Learning Tutorials

Deep Learning Tutorials

This topic is for readers who want neural networks to be inspectable instead of magical. The route covers matrix calculus, backpropagation, convolution, receptive fields, optimizer geometry, attention, and small CNN projects. Each page tries to connect a formula, a tensor shape, and a runnable implementation so that the reader can debug the computation rather than only memorize model names.

Frameworks make deep learning convenient, but they also hide the failure points that matter in real work: mismatched shapes, unstable learning rates, incorrect labels, leaking validation data, and gradients that do not flow as expected. The pages in this hub keep the examples small enough that the intermediate values can still be reasoned about.

Suggested Path

Start with matrix calculus and the two-layer MLP backpropagation article if you are still building the connection between derivatives and code. Then move to convolution and receptive field math, where padding, stride, channels, and im2col show how image models are assembled from simple operations. After that, the attention and KV-cache material will be easier to read because the tensor bookkeeping is already familiar.

For every article, change one parameter and rerun the example. A different learning rate, padding value, initialization seed, or hidden width can reveal why the original configuration worked. That habit is more valuable than copying the final code unchanged.

What This Hub Does Not Claim

These tutorials are educational, not production training recipes. They do not guarantee state-of-the-art accuracy, large-scale efficiency, or suitability for sensitive deployments. The purpose is to build a reliable mental model of the computation so that larger frameworks and papers become easier to evaluate later.

Debugging Evidence

A deep learning article is more useful when it shows what can be inspected. For a backpropagation example, that may be a forward value, loss, local derivative, and gradient update. For a convolution example, it may be the input shape, kernel shape, padding, stride, output shape, and receptive field. For attention, it may be token positions, query/key/value shapes, scores, softmax weights, and cached state.

Those intermediate values matter because most mistakes are not visible in the final loss alone. A model can train slowly because the learning rate is wrong, the target is shifted, the shape is transposed, or the validation split leaks information. The route encourages readers to check the computation step by step before treating the model as a black box.

From Small Models to Larger Systems

The small examples on this site are stepping stones. They do not replace modern frameworks, but they make framework behavior easier to question. Once the reader understands a two-layer MLP or a tiny CNN, larger systems can be evaluated with better questions: which tensor is being normalized, which parameters are being updated, what is cached, what is frozen, and what evidence shows the model is generalizing?

This also keeps the page grounded in original learning value. The site is not trying to summarize every popular model name. It focuses on computations that a reader can trace, modify, and compare with code. That is the foundation for reading papers, using libraries responsibly, and debugging larger training workflows later.

Computation Evidence Matrix

Topic Core question Intermediate values to inspect
Backpropagation Does the gradient flow through the intended path? Forward output, loss, local derivative, weight gradient, and update size.
Convolution and receptive field Do output shape and receptive field match the derivation? Input shape, kernel, padding, stride, output shape, and covered region.
Optimizer behavior Are parameter updates controlled by a reasonable learning rate? Gradient norm, step size, loss curve, oscillation range, and stopping rule.
Attention Which tokens are compared, and is cached state consistent? Q/K/V shapes, scores, softmax weights, mask, and KV-cache state.

Topic hub

Deep Learning Math and CNN Tutorials

A focused learning hub for neural network basics, matrix calculus, backpropagation, optimizers, convolution, attention, and CIFAR-10 Tiny CNN.

Built for readers searching for deep learning math, backpropagation derivations, convolution formulas, attention calculation, CIFAR-10 CNN tutorials, and visual experiments.

Open AI learning path Open resources Copy share links

Editorial notes

Why these articles belong in one route

This hub is not a glossary of deep learning terms. Each concept is tied to a checkable computation: what tensors the forward pass creates, how the loss sends error signals back to parameters, and how the optimizer changes the next update.

The route starts with small matrices and hand-calculated gradients, then moves into convolution, attention, and the CIFAR-10 Tiny CNN. Readers can verify that formulas and code output agree before moving to larger framework projects.

The companion assets include runnable scripts, animation, and training output, so review can cross-check formulas, source code, and measured results instead of stopping at a short conceptual summary.

What you will build

You will hand-calculate core formulas, run NumPy experiments, inspect figures and animations, then move into a small CNN image classifier.

  • deep learning math visualized
  • backpropagation hand calculation
  • Adam optimizer geometry
  • convolution receptive field formula
  • Transformer attention hand calculation

Recommended reading order

Start with concepts, then move into runnable projects

Neural Network Basics

Move from perceptrons to activation, forward propagation, backpropagation, and training loops.

Level: Intermediate Reading time: 8 min
  • Neural Networks
  • Backpropagation
  • Python

Matrix Calculus for Neural Networks

Derive dL/dW for y = Wx + b and verify it with finite differences.

Level: Intermediate Reading time: 13 min
  • Matrix Calculus
  • NumPy
  • Gradient Check

Backpropagation as a Computation Graph

Trace local gradients through ReLU and softmax cross-entropy in a two-layer MLP.

Level: Intermediate Reading time: 14 min
  • Backpropagation
  • Computation Graph
  • Softmax

Gradient Descent and Optimizer Geometry

Compare gradient descent, momentum, and Adam on a visible quadratic loss surface.

Level: Intermediate Reading time: 13 min
  • Gradient Descent
  • Momentum
  • Adam
  • Loss Surface

Convolution and Receptive Field Math

Compute convolution output size, receptive fields, channel mixing, and im2col layout.

Level: Intermediate Reading time: 13 min
  • Convolution
  • Receptive Field
  • im2col

Transformer Attention Math

Hand-calculate Q/K/V scores, softmax weights, masks, multi-head structure, and KV cache.

Level: Intermediate Reading time: 14 min
  • Transformer
  • Attention
  • QKV
  • KV Cache

CIFAR-10 Tiny CNN Tutorial in C

Build and train a small convolutional neural network for CIFAR-10 image classification, then read its loss and accuracy output.

Level: Intermediate Reading time: 13 min
  • C
  • CNN
  • CIFAR-10
  • Backpropagation

Model Training and Evaluation

Understand loss, overfitting, train/test splits, accuracy, recall, and F1.

Level: Beginner Reading time: 9 min
  • Model Training
  • Metrics
  • Evaluation

Resources and distribution assets

Code, data, diagrams, and share assets in one place

FAQ

Direct answers to common search questions

Is this hub suitable without prior deep learning experience?

Yes, if you have programming basics. Start with neural network basics, then continue through matrix calculus, backpropagation, optimizers, convolution, and attention.

Why use C for the first CNN instead of only PyTorch?

The C version exposes data layout, convolution loops, and parameter updates, which makes it useful before relying on a framework.

Scroll down