Skip to content
barhancPublic

About

A minimal neural network library in NumPy, born out of experiments during the Neural Networks and Deep Learning course at AGH University

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

181 Commits

Folders and files

Repository files navigation

tinny

uv Python 3.12+ NumPy License: MIT

A minimal, educational neural network and energy-based model library implemented from scratch in pure NumPy.

tinny was created to demystify deep learning internals: no opaque autograd magic, no heavy framework bloat — just explicit mathematical operations, vectorized tensor calculus, and property-tested analytical gradient backpropagation. See the notebooks/ directory for experiment implementations (including MLPs, CNNs, Autoencoders, GANs, and DBNs) along with solutions and commentary for course assignments.

Core Highlights

  • Pure NumPy Core: Transparent matrix implementations of forward and backward passes with zero black-box dependencies.
  • Energy-Based Models (EBMs): Complete implementations of Restricted Boltzmann Machines (RBMs) and Deep Belief Networks (DBNs), trained via Contrastive Divergence ($CD_k$) and Persistent Contrastive Divergence (PCD).
  • Extensible Tensor Layer API: Layers are defined as parametrized tensor functions $y = f(x; \theta)$ supporting sequential composition and residual skip connections (Residual).
  • Comprehensive Layer Suite: Dense (Linear), 2D Convolution (Conv2D with vectorized im2col), MaxPool2D, Flatten, Dropout, and BatchNorm.
  • Rigorously Tested Gradients: Analytical gradients are validated against numerical finite differences via property-based testing with Hypothesis and pytest.
  • Accompanying Theory Notes: Mathematical derivations and conceptual notes available at barhanc.github.io/notes.

Quickstart

Ensure you have uv installed.

1. Installation

git clone https://github.com/barhanc/tinny.git
cd tinny

# Synchronize the environment and install dependencies in seconds
uv sync

2. Building & Training a Model

from tinny import nn
from tinny.utils import softmax, chunks

# Define a Multi-Layer Perceptron
model = nn.Sequential(
    nn.Linear(vsize=784, hsize=128),
    nn.ReLU(),
    nn.Dropout(p=0.2),
    nn.Linear(vsize=128, hsize=10),
)

optimizer = nn.Adam(model.parameters(), lr=1e-3)

# Training step across mini-batches
for X_batch, y_batch in zip(chunks(X_train, 128), chunks(y_train, 128)):
    # Forward pass
    logits = model.forward(X_batch, training=True)

    # Compute cross-entropy loss gradient
    grad_y = (softmax(logits) - y_batch) / len(X_batch)

    # Backward pass & parameter update
    model.backward(grad_y)
    optimizer.apply(model.gradients())

About

A minimal neural network library in NumPy, born out of experiments during the Neural Networks and Deep Learning course at AGH University

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages