FusionBench Project Template is designed to help researchers and developers quickly set up a new project for deep model fusion using PyTorch and FusionBench. This template provides a well-organized project structure, pre-configured settings, and essential tools to streamline the development process. Key features include:

• Hydra Configuration: Simplifies the management of complex configurations.
• PyTorch Lightning: Facilitates the training and evaluation of models.
• Pre-configured Linters: Ensures code quality with tools like Black and isort.
• Jupyter Notebooks: Supports interactive development and experimentation.
• Modular Design: Encourages clean and maintainable code with a clear separation of concerns.

By using this template, you can focus on developing and experimenting with your fusion models without worrying about the initial setup and configuration.

The directory structure of new project looks like this:

├── .github <- Github Actions workflows ├── .vscode <- VSCode settings ├── config <- Hydra configs │ ├── hydra <- Hydra configs │ ├── fabric <- Fabric configs │ ├── fabric_logger <- Fabric logger configs │ ├── method <- Method configs │ ├── modelpool <- Model pool configs │ ├── taskpool <- Task pool configs │ │ │ └── main.yaml <- Main config for training │ ├── notebooks <- Jupyter notebooks. Here you can place | some example code and notebooks. | ├── scripts <- Shell scripts │ ├── src <- Source code │ ├── data <- Data classes │ ├── models <- Model classes │ ├── method <- Method classes │ ├── modelpool <- Model pool classes | ├── taskpool <- Task pool classes │ └── utils <- Utility functions | ├── tests <- Tests of any kind │ ├── .gitignore <- List of files ignored by git ├── requirements.txt <- File for installing python dependencies ├── setup.py <- File for installing project as a package └── README.md 

FusionBench follows a three-component architecture to perform model fusion experiments:

graph LR CLI[fusion_bench CLI] --> Hydra[Hydra Config] Hydra --> Program[Program] Program --> MP[ModelPool<br/>Manages Models<br/>& Datasets] Program --> Method[Method<br/>Fusion Algorithm] Program --> TP[TaskPool<br/>Evaluation Tasks] MP --> Method Method --> Merged[Merged Model] Merged --> TP TP --> Report[Evaluation Report] style CLI fill:#e1f5e1 style Hydra fill:#f0e1ff style Method fill:#ffe1f0 style Merged fill:#fff4e1 style Report fill:#e1f0ff 

Key Components:

1. CLI: Entry point using Hydra for configuration management
2. Program: Orchestrates the fusion workflow (e.g., FabricModelFusionProgram)
3. ModelPool: Manages task-specific models and their datasets
4. Method: Implements the fusion algorithm (e.g., Simple Average, Task Arithmetic, AdaMerging)
5. TaskPool: Evaluates the merged model on benchmark tasks

Workflow Steps:

1. User runs fusion_bench with config overrides
2. Hydra loads YAML configs for method, modelpool, and taskpool
3. Program instantiates all three components
4. Method executes fusion algorithm on ModelPool
5. TaskPool evaluates the merged model
6. Results are saved and reported

This template is motivated by Lightning-Hydra-Template.

What it does

# clone project git clone https://github.com/YourGithubName/your-repo-name cd your-repo-name # [OPTIONAL] create conda environment conda create -n myenv python=3.12 # recommend python 3.12+ conda activate myenv # install pytorch according to instructions # https://pytorch.org/get-started/ # install requirements pip install -r requirements.txt # install project as editable package (edit pyproject.toml as needed) pip install -e .

# clone project git clone https://github.com/YourGithubName/your-repo-name cd your-repo-name # create conda environment and install dependencies conda env create -n myenv python=3.12 # activate conda environment conda activate myenv

run my method

fusion_bench \ --config-name main \ # method=... # method.option_1=... # modelpool=... # ...

or

bash scripts/run_experiments.sh

This project is based on FusionBench and created using FusionBench Project Template.