Skip to content

Support dense Qwen3.5 model architecture (qwen3_5) #31

@jsirish

Description

@jsirish

Description

The dense Qwen3.5 architecture (model_type: qwen3_5) is not currently supported by AF-M. Only the MoE variant (qwen3_5_moe) works.

Steps to Reproduce

afm mlx -m mlx-community/Qwen3.5-27B-8bit -p 8001

Error

Error starting MLX server: loadFailed("mlx-community/Qwen3.5-27B-8bit: Unsupported model type: qwen3_5") 

Expected Behavior

AF-M should load dense Qwen3.5 models (27B, etc.) the same way it loads the MoE variants (122B-A10B, 35B-A3B).

Environment

  • AF-M version: v0.9.5-cd2941e (afm-next via Homebrew)
  • macOS 26 on Mac Studio M3 Ultra
  • Model: mlx-community/Qwen3.5-27B-8bit (config.json has model_type: qwen3_5, architectures: ['Qwen3_5ForConditionalGeneration'])

Notes

The MoE variants work perfectly — Qwen3.5-122B-A10B-4bit and Qwen3.5-35B-A3B-4bit both load and run well. The Scripts/patches/ directory has Qwen3_5MoE.swift but no corresponding Qwen3_5.swift for the dense architecture.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions