Recreating every milestone in Machine Learning and Artificial Intelligence — from Transformers to Perceptrons.
ReplicateAI is an open initiative to rebuild and verify every major paper in ML/AI history,
starting from modern foundation models (2023–2025) and tracing backward to the origins of AI.
We believe that understanding AI means rebuilding it — line by line, layer by layer.
“Because science means reproducibility.”
- 📜 Goal: Faithfully re-implement influential ML/AI papers with open code, datasets, and experiments
- 🧱 Scope: From Qwen2.5 (2025) to Perceptron (1958)
- 🧠 Approach: Reverse timeline — start with Foundation Models, then trace history backward
- 🧾 Output: Each paper becomes a self-contained, reproducible module with reports and experiments
The golden age of open-source foundation models.
| Year | Paper / Model | Organization | Why It Matters | Replicate Goal | Status |
|---|---|---|---|---|---|
| 2025 | Qwen2.5 | Alibaba | Fully open multimodal model (text + image) | Rebuild text/image pipeline | 🧭 Planned |
| 2025 | DeepSeek-V2 | DeepSeek | MoE + RLHF efficiency breakthrough | Replicate expert routing and reward pipeline | 🧭 Planned |
| 2025 | Claude 3 Family | Anthropic | Leading alignment via Constitutional AI | Explore rule-based alignment principles | 🧭 Planned |
| 2024 | LLaMA 3 | Meta | Open foundation model standard | Implement scaled transformer + tokenizer | 🧭 Planned |
| 2024 | Mixtral 8×7B | Mistral | Sparse Mixture-of-Experts architecture | Implement routing + expert parallelism | 🧭 Planned |
| 2024 | Phi-2 / Phi-3 | Microsoft | Small but high-quality model; data-centric | Rebuild synthetic data pipeline | 🧭 Planned |
| 2024 | Gemini 1 / 1.5 | Google DeepMind | Vision + Text + Reasoning | Prototype multimodal reasoning pipeline | 🧭 Planned |
| 2023 | Qwen-VL | Alibaba | Vision-language alignment model | Replicate visual encoder + text fusion | 🧭 Planned |
| 2023 | BLIP-2 / MiniGPT-4 | Salesforce / HKU | Lightweight multimodal bridging | Implement pretrain connector | 🧭 Planned |
| 2023 | LLaMA 1 / 2 | Meta | Open LLM baseline | Implement tokenizer + attention stack | 🧭 Planned |
| Year | Paper | Author | Goal | Status |
|---|---|---|---|---|
| 2021 | CLIP | Radford, et al. | Align Vision and NLP in same space using contrastive learning | 🔬 Replicating |
| 2020 | ViT | Dosovitskiy et al. | Vision Transformer | ✅ Done |
| 2018 | BERT | Devlin et al. | Masked Language Modeling | 🔬 Replicating |
| 2017 | Transformer | Vaswani et al. | “Attention Is All You Need” | ✅ Done |
| 2014 | Seq2Seq | Sutskever et al. | Encoder-decoder translation | 🧭 Planned |
| 2013 | Word2Vec | Mikolov et al. | Learn word embeddings | 🧭 Planned |
| 2015 | Bahdanau Attention | Bahdanau et al. | RNN + Attention | 🧭 Planned |
| Year | Paper | Author | Goal | Status |
|---|---|---|---|---|
| 2015 | ResNet | He et al. | Residual learning | 🧭 Planned |
| 2014 | VGG | Simonyan et al. | Deep CNN architectures | 🧭 Planned |
| 2012 | AlexNet | Krizhevsky et al. | GPU-based CNN | 🧭 Planned |
| 2006 | DBN / RBM | Hinton | Layer-wise pretraining | 🧭 Planned |
| Year | Paper | Author | Goal | Status |
|---|---|---|---|---|
| 2001 | Random Forests | Breiman | Ensemble learning | 🧭 Planned |
| 1997 | AdaBoost | Freund & Schapire | Boosting algorithms | 🧭 Planned |
| 1995 | SVM | Vapnik | Maximum margin classifier | 🧭 Planned |
| 1977 | EM Algorithm | Dempster et al. | Expectation-Maximization | 🧭 Planned |
| Year | Paper | Author | Goal | Status |
|---|---|---|---|---|
| 1986 | Backpropagation | Rumelhart et al. | Gradient-based learning | 🧭 Planned |
| 1985 | Boltzmann Machine | Hinton et al. | Generative stochastic model | 🧭 Planned |
| 1982 | Hopfield Network | Hopfield | Associative memory | 🧭 Planned |
| 1958 | Perceptron | Rosenblatt | Linear separability | 🧭 Planned |
🧭 Planned ↓ 🔬 In Reproduction ↓ 🧪 Under Evaluation ↓ 📈 Verified ↓ 🧾 Documented ↓ 🧰 Extended (optional) ReplicateAI/ ├── stage1_foundation/ │ ├── 2025_Qwen2.5/ │ ├── 2024_LLaMA3/ │ └── 2023_CLIP/ ├── stage2_representation/ │ ├── 2018_BERT/ │ ├── 2017_Transformer/ │ └── 2013_Word2Vec/ ├── stage3_deep_renaissance/ │ ├── 2015_ResNet/ │ ├── 2012_AlexNet/ │ └── 2006_DBN/ ├── stage4_statistical/ │ ├── 2001_RandomForest/ │ └── 1995_SVM/ └── stage5_foundations/ ├── 1986_Backprop/ └── 1958_Perceptron/ Each paper module includes:
📄 README.md — Paper summary & objective 📘 report.md — Reproduction results & analysis 📓 notebook/ — Interactive demo 💻 src/ — Core implementation 🔗 references.bib — Original citation We welcome contributions from researchers, engineers, and students who believe in reproducibility.
- Fork the repo
- Pick a paper or model not yet implemented
- Follow the Paper Template
- Submit a PR with your code and report
✅ Please include:
- clear code (PyTorch / JAX / NumPy)
- short experiment or visualization
- reproducibility notes or deviations
| Stage | Era | Progress |
|---|---|---|
| 🪐 Foundation (2023–2025) | Modern LLM & Multimodal | ░░░░░░░░░░░░░░ 0% |
| 🔍 Representation (2013–2020) | Transformers & Embeddings | ░░░░░░░░░░░░░░ 0% |
| 🧩 Deep Renaissance (2006–2014) | CNN Era | ░░░░░░░░░░░░░░ 0% |
| 📊 Statistical (1990s–2000s) | Classical ML | ░░░░░░░░░░░░░░ 0% |
| 🧬 Foundations (1950s–1980s) | Neural Origins | ░░░░░░░░░░░░░░ 0% |
If you use or reference this project, please cite:
@misc{replicateai2025, author = {ReplicateAI Contributors}, title = {ReplicateAI: Rebuilding the History of Machine Learning and Artificial Intelligence}, year = {2025}, url = {https://github.com/duoan/ReplicateAI} }“Replicate. Verify. Understand.”
⭐️ Star this repo if you believe reproducibility is the foundation of true intelligence.