A simple WebUI to chat with Orpheus TTS via WebRTC.
This project requires two external OpenAI-compatible endpoints:
- Text generation: Connect and chat with your favorite LLM.
- Orpheus TTS model: Serve the fine-tuned Orpheus model. As Orpheus-3B is just a fine-tune of llama 3.2 3B, you can easily serve it with llama.cpp or vllm.
$ llama-server --port 11434 --model gemma-3-12b-it-Q8_0.gguf # LLM $ llama-server --port 8080 --model orpheus-3b-0.1-ft-q8_0.gguf # Orpheus# Set up LLM endpoint $ export OPENAI_BASE_URL=http://localhost:11434 $ export OPENAI_API_KEY=dummy $ export OPENAI_MODEL=model # Set up Orpheus endpoint $ export ORPHEUS_BASE_URL=http://localhost:8080 $ export ORPHEUS_API_KEY=dummy # Provide HF token if you need a TURN server for WebRTC to traverse NAT. # See: https://fastrtc.org/deployment/#community-server $ export HF_TURN_TOKEN=hf-******* # Run the app. $ uv run python -m src.orpheus-chat-webui