This is my personal space to learn different ML algorithms by specifying, implementing, and testing them.

src/ ├── fine_tuning/ │ ├── imag_transformer_finetune.py # fine-tuning for vision transformers │ └── llm_finetuning.py # fine-tuning for large language models │ ├── general_algorithms/ │ ├── linear_regression.py # Linear regression (closed-form, GD, SGD) (tested) │ └── logistic_regression.py # Logistic regression (GD, SGD) (tested) │ └── neural_networks/ ├── simple_NNs.py # Logistic regression NN, MLPs, regularized NNs ├── CNN.py # Convolutional Neural Networks ├── RNN.py # Recurrent Neural Networks └── GPT.py # Transformer decoder (GPT-like) 

Tests live under test/ and validate the correctness of each algorithm. Additional tests are being written accordingly when time permits.

• Linear Regression (tested)

  • Closed-form solution (Normal Equation / np.linalg.lstsq)
  • Gradient Descent (GD)
  • Stochastic Gradient Descent (SGD)

• Logistic Regression (tested)

  • Binary classification with sigmoid activation
  • Cross-entropy loss, GD and SGD variants

• Simple NNs (tested)

  • Logistic Regression (PyTorch)
  • Single-layer MLP
  • Deep Neural Networks (DNNs)
  • Variants with L1, L2, ElasticNet regularization and Dropout

• CNNs: Convolutional layers, pooling, activation pipelines

• RNNs: Basic recurrent architectures for sequence data

• GPT: Decoder-only transformer blocks

• LLM fine-tuning: pipelines for adapting pre-trained LLMs
• Vision Transformer fine-tuning: adapting image transformers on downstream tasks

All modules will be covered with pytest test suites, and tests are being added when time permits

• ✅ Linear & logistic regression (NumPy): implemented and tested
• ✅ Simple neural networks (MLPs, regularization helpers): implemented, tests in progress
• ⚠️ CNN / RNN / GPT modules: experimental / work in progress
• 🚧 Fine-tuning scripts: initial prototypes for future expansion

You are welcome to use this repo for ethical and scientific purposes. Clone the repository by running;

git clone https://github.com/soghomon-n/ml-algorithms-implementation.git

Dependencies:

• Python 3.9+
• numpy
• torch
• pytest (for testing)

All algorithms expect inputs and targets as numpy.ndarray objects. For neural network modules implemented with PyTorch, tensors are internally converted to appropriate floating-point types where necessary.

from src.general_algorithms.linear_regression import LinearRegressionGD import numpy as np X = np.random.randn(200, 3) w_true = np.array([1.5, -2.0, 0.7]) y = X @ w_true + 0.1 * np.random.randn(200) model = LinearRegressionGD(lr=0.01, n_iters=1000) model.fit(X, y) print("Learned weights:", model.w) print("MSE:", model.mse(X, y))

• Build core machine learning algorithms from scratch to help me understand them more.
• Provide PyTorch equivalents for neural network architectures.
• Ensure correctness through unit testing.
• Extend into advanced deep learning architectures, including CNNs, RNNs, and Transformers with fine-tuning.

MIT License © 2025