Synapse-NN
A minimal neural network library built to understand how deep learning works internally.
# pip install synapse-nn # Use of virtual environments is suggested from synapse import Matrix, Dense x = Matrix([[1, 2]]) layer = Dense(2, 4) output = layer.forward(x).relu() print(output)
About the Library - version 0.1.1
Synapse-NN is designed for learning how neural networks work internally. It implements matrix operations, fully connected layers, and activation functions without using external ML libraries.
It helps you understand: matrix multiplication, forward propagation, bias addition, and activation flow.
Matrix Class
Matrix Creation
The Matrix class is the core data structure of the library. It stores data as a 2D list and supports mathematical operations like addition, subtraction, and multiplication.
Matrix([[1, 2], [3, 4]])
Example use case: initializing dataset inputs or weights manually.
Shape Property
The shape property returns the size of the matrix in terms of rows and columns. It is important for validating matrix operations like dot product.
x.shape
Example output: (2, 3)
Addition
Matrix addition performs element-wise addition. Both matrices must have the same shape.
A + B
Example: A = [[1,2], B = [[3,4]] Result = [[4,6]]
Subtraction
Element-wise subtraction between two matrices of equal shape. Used in gradient-like computations.
A - B
Scalar Multiplication
Multiplies every element of the matrix by a scalar value. Useful for scaling weights or gradients.
A * 2 2 * A
Dot Product
This is the most important operation in neural networks. It performs matrix multiplication between two matrices. Used in Dense layers for forward propagation.
A.dot(B)
Example: Input × Weights = Output
Transpose
Swaps rows and columns of a matrix. Used to align dimensions for multiplication.
A.transpose()
Dense Layer
What is Dense Layer
A Dense layer is a fully connected neural network layer where every input neuron is connected to every output neuron.
Initialization
Creates a layer with random weights and zero biases.
Dense(input_size, output_size)
Forward Pass
Computes output using: output = input · weights + bias
layer.forward(x)
This is the core computation step of a neural network.
Activation Functions
ReLU
ReLU removes negative values and keeps positive values unchanged. It introduces non-linearity into the model.
A.relu()
Example: [-2, 3, -1] → [0, 3, 0]
Sigmoid
Maps values between 0 and 1. Used in binary classification problems.
A.sigmoid()
Tanh
Maps values between -1 and 1. Centered activation useful in hidden layers.
A.tanh()
Basic Example
from synapse import Matrix, Dense x = Matrix([[1, 2]]) layer = Dense(2, 4) output = layer.forward(x).relu() print(output)
More Examples
Example 1: Simple Forward Pass
x = Matrix([[2, 3]]) layer = Dense(2, 2) print(layer.forward(x))
Example 2: Activation Flow
x = Matrix([[1, -2]]) layer = Dense(2, 2) output = layer.forward(x).relu() print(output)
Example 3: Matrix Operations
A = Matrix([[1, 2]]) B = Matrix([[3, 4]]) print(A + B) print(A.dot(B.transpose()))
Creator
About the Creator
Name: Yashdeep
Synapse-NN was created as a learning project to explore the foundations of neural networks, matrix mathematics, and deep learning systems. The long-term goal is to expand it into a complete educational neural network framework featuring backpropagation, loss functions, optimizers, and model training.
Current Version: v0.1.1
Status: Active Development
Project: Synapse-NN