CIFAR-10 Tiny CNN Tutorial in C: Build and Train a Small Convolutional Neural Network

This is a CIFAR-10 Tiny CNN tutorial in C. By the end, you can build and train a small convolutional neural network for CIFAR-10 image classification: load the binary dataset, run 3 x 3 convolution, ReLU, 2 x 2 max pooling, fully connected logits, softmax, and one small training run.

This article is based on the local cifar10_tiny_cnn.c project. The goal is not to reach state-of-the-art CIFAR-10 accuracy. The goal is to make each part of a CNN visible as code you can inspect, compile, and modify.

If you are still learning the foundations, start with Neural Network Basics. If you have already read the handwritten digit softmax classifier in C, this article is the next step from linear classification into convolutional image models.

1. What The CIFAR-10 Input Looks Like

Each CIFAR-10 image is a 32 x 32 color image with red, green, and blue channels. The input shape in the program is therefore 3 x 32 x 32. Each sample has one label from 10 classes.

The source fixes that shape with a few constants:

#define IMG_C 3
#define IMG_H 32
#define IMG_W 32
#define NCLASS 10

This is different from the handwritten digit softmax project. The digit project flattens a grayscale image into one vector. The CNN keeps the spatial layout first, so convolution filters can scan local regions and detect color, edge, and texture patterns.

2. Tiny CNN Architecture And Parameter Count

The current code uses 16 filters of size 3 x 3. Because the convolution is valid convolution, a 32 x 32 input becomes 30 x 30 after convolution. Then 2 x 2 max pooling reduces it to 15 x 15.

#define NF 16
#define K 3
#define CONV_H (IMG_H - K + 1)
#define CONV_W (IMG_W - K + 1)
#define POOL_H (CONV_H / 2)
#define POOL_W (CONV_W / 2)
#define FEAT (NF * POOL_H * POOL_W)

The full path is:

• Input: 3 x 32 x 32
• Convolution: 16 filters, 3 x 3, output 16 x 30 x 30
• ReLU: clips negative activations to 0
• Max pooling: 2 x 2, output 16 x 15 x 15
• Fully connected layer: maps pooled features to 10 logits
• Softmax: converts logits into class probabilities

The model is intentionally small. The convolution layer has 16 * 3 * 3 * 3 + 16 = 448 parameters. The fully connected layer has 10 * 3600 + 10 = 36010 parameters. The total is 36458 parameters, which is useful for learning and debugging rather than competing with modern CIFAR-10 models.

3. Convolution Written Directly In C

The convolution layer is nested loops. Each filter scans every output location, then multiplies and accumulates over the 3 input channels and the 3 x 3 local window.

float sum = net->conv_b[f];
for (int c = 0; c < IMG_C; c++) {
    for (int r = 0; r < K; r++) {
        for (int t = 0; t < K; t++) {
            sum += net->conv_w[f][c][r][t] *
                   get_pixel(s->x, c, i + r, j + t);
        }
    }
}
conv[f][i][j] = sum > 0.0f ? sum : 0.0f;

The last line also applies ReLU. This plain implementation is useful because it exposes the relationship between filters, channels, local windows, and output positions.

4. Max Pooling Keeps The Strongest Local Response

The pooling layer compresses every 2 x 2 window into one value by keeping the maximum activation. The code also records where the maximum came from, because backpropagation only sends the gradient back to that winning position.

float best = conv[f][base_i][base_j];
int best_idx = 0;
for (int di = 0; di < 2; di++) {
    for (int dj = 0; dj < 2; dj++) {
        float v = conv[f][base_i + di][base_j + dj];
        if (v > best) {
            best = v;
            best_idx = di * 2 + dj;
        }
    }
}

This reduces the feature map size and gives the model a small amount of local translation tolerance.

5. Fully Connected Layer And Softmax

The pooled features are flattened into one vector and passed into a fully connected layer. Each class has its own weights, producing 10 logits.

for (int k = 0; k < NCLASS; k++) {
    float z = net->fc_b[k];
    for (int p = 0; p < FEAT; p++) {
        z += net->fc_w[k][p] * feat[p];
    }
    logits[k] = z;
}

Softmax normalizes these raw scores into probabilities. Training uses cross-entropy loss, and prediction selects the class with the highest probability.

6. What Backpropagation Updates

This program does not hide backpropagation inside a framework. It explicitly does four things:

• subtracts the true label from the softmax probabilities to get output gradients
• updates fully connected weights and biases
• sends gradients back through max pooling and ReLU
• updates every convolution filter from the matching input window

net->fc_w[k][p] -= LR * dlogits[k] * feat[p];
net->fc_b[k] -= LR * dlogits[k];
...
net->conv_w[f][c][r][t] -= LR * dw;
net->conv_b[f] -= LR * db;

This is the most valuable part of the project. CNN training is not magic: each layer computes its gradients and moves its parameters in a direction that lowers the loss.

7. Compile And Run Locally

The site publishes the source file, sample weights, sample predictions, and explanation notes. It does not publish the full CIFAR-10 dataset. Download the CIFAR-10 binary version from the official source, extract cifar-10-batches-bin, and run the program against that folder.

gcc -O2 -std=c11 cifar10_tiny_cnn.c -lm -o cifar10_tiny_cnn
./cifar10_tiny_cnn ./cifar-10-batches-bin 1 2000 1000

The four command-line arguments are:

• data directory
• number of epochs
• training sample limit
• test sample limit

Start with a small sample limit first. After the full flow works, increase the training sample count and epoch count gradually.

8. Training Results: Loss And Accuracy

I ran the small-sample command with 2000 training images, 1000 test images, and 1 epoch. The real output was:

Loaded train=2000, test=1000
epoch 1 step 1000/2000 loss=2.0507 train_acc_recent_est=0.809
epoch 1 step 2000/2000 loss=1.9440 train_acc_recent_est=0.835
epoch 1 done: avg_loss=1.9440 train_acc=0.835 test_acc=0.284
Model saved to model_weights.bin
Predictions saved to test_predictions.csv

Do not read this as a high-performance result. The training accuracy estimate is much higher than the test accuracy, which landed at 0.284 in this small run. That is expected for a tiny educational CNN trained on a small slice for one epoch. The point is to verify the full pipeline and make the computations readable.

9. Reading The Output Files

The program writes two output files after a run:

• model_weights.bin: saved model parameters from one local run
• test_predictions.csv: prediction output for checking format and class distribution

The resource library only hosts small companion materials. The full training data stays with the official CIFAR-10 source instead of being duplicated on this site.

10. How To Improve The CIFAR-10 Tiny CNN

If your goal is higher accuracy rather than understanding the basic CNN pipeline, improve the project in this order:

• Train on more data: raise the training limit from 2000 to 10000, 20000, or the full training set
• Use more epochs: one epoch only proves that the pipeline runs
• Add mini-batches: the current code updates on one sample at a time
• Add data augmentation: random crops, horizontal flips, and color jitter help CIFAR-10 generalization
• Use a deeper model: add another convolution layer or increase the channel count
• Port it to PyTorch: if engineering speed matters, keep this C version as the reference and rebuild it in PyTorch

11. Project Boundaries

This tiny CNN is an educational implementation, not a production image classifier. Its limits are clear:

• pure C single-sample training is much slower than modern deep learning frameworks
• there is no mini-batching, data augmentation, regularization, or learning-rate schedule
• the network is intentionally small, so accuracy will not match modern CIFAR-10 models
• the main goal is to understand CNN forward propagation and backpropagation

If you have already read the handwritten digit softmax project, this article is the next step: moving from a linear classifier into an image model with local receptive fields.

12. CIFAR-10 Experiment Audit Table

The goal of this tiny CNN is not leaderboard performance. It is to make convolution, pooling, softmax, and backpropagation inspectable. The audit table below helps readers decide what one run proves and what it does not prove.

13. FAQ

Is this a PyTorch CIFAR-10 tutorial?

No. This article intentionally uses C so the convolution, pooling, softmax, and backpropagation steps are visible. A PyTorch version should be a separate implementation article.

Why is the test accuracy low?

The example uses a very small CNN, 2000 training samples, and 1 epoch. It is an educational implementation, not a tuned CIFAR-10 model.

Where do I get the full CIFAR-10 dataset?

The site does not mirror the full dataset. Download the CIFAR-10 binary version from the official source, extract cifar-10-batches-bin, and pass that folder to the program.

How does this connect to the handwritten digit project?

The handwritten digit dataset article and softmax classifier article explain linear image classification first. This article adds local receptive fields, convolution filters, and pooling.

14. Companion Resources And Next Reading

The resource library includes the C source, sample model weights, sample predictions, and CNN explanation PDF. Download the full CIFAR-10 dataset from its official source when running the project locally.

A good reading path is: neural network basics, then the C softmax classifier, then this CIFAR-10 CNN tutorial. The related files are also collected in the resource library.