Skip to content

fix: pass num_workers from ccproxy.yaml to litellm CLI#12

Open
npinto wants to merge 1 commit intostarbaser:mainfrom
npinto:fix/num-workers-silent-ignore
Open

fix: pass num_workers from ccproxy.yaml to litellm CLI#12
npinto wants to merge 1 commit intostarbaser:mainfrom
npinto:fix/num-workers-silent-ignore

Conversation

@npinto
Copy link
Copy Markdown

@npinto npinto commented Feb 13, 2026

Bug

The litellm.num_workers setting in ccproxy.yaml is silently ignored. start_litellm() builds the litellm command as litellm --config <path> without reading num_workers from the config, so the proxy always starts with litellm's default worker count.

Template suggests it works:

# ccproxy/templates/ccproxy.yaml litellm: num_workers: 4 # ← never read, never passed

Actual command (before fix):

litellm --config /path/to/config.yaml 

Expected command (after fix):

litellm --config /path/to/config.yaml --num_workers 4 

Fix

Read num_workers from the litellm: section of ccproxy.yaml and append --num_workers N to the litellm subprocess command when configured. When omitted, litellm uses its own default (CPU count or 4).

9 lines in src/ccproxy/cli.py.

Test

One unit test following the existing @patch("subprocess.run") pattern in test_cli.py: write a ccproxy.yaml with num_workers: 8, call start_litellm(), assert --num_workers 8 appears in the subprocess command.

Fails on current HEAD, passes with fix. Zero regressions on existing tests.

The `litellm.num_workers` setting in `ccproxy.yaml` was silently ignored. `start_litellm()` built the litellm command without reading this value, so the proxy always started with litellm's default worker count regardless of what was configured. Read `num_workers` from the `litellm:` section of `ccproxy.yaml` and pass it as `--num_workers` to the litellm subprocess when set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant