mirror of
https://github.com/mark3labs/kit.git
synced 2026-06-13 19:20:06 +00:00
fix(cmd): route prefixed models through custom wire when --provider-url is set
When --provider-url was set with an explicit --model that already carried a provider prefix (e.g. google/gemma-4-12b served by LM Studio), Kit honored the prefix and routed through the Google wire protocol instead of the user-supplied endpoint, producing confusing upstream errors. - Strip any non-custom provider prefix from --model when --provider-url is set, so the request always lands on the OpenAI-compatible custom wire pointed at the user's URL. - Leave behavior unchanged when --provider-url is absent. - Document the rewrite in www/pages/providers.md.
This commit is contained in:
+19
-4
@@ -735,12 +735,27 @@ func runNormalMode(ctx context.Context) error {
|
||||
viper.Set("model", "custom/custom")
|
||||
}
|
||||
|
||||
// When --provider-url is set with an explicit --model that lacks a provider
|
||||
// prefix (no "/"), auto-prefix with "custom/" for OpenAI-compatible endpoints.
|
||||
// When --provider-url is set with an explicit --model, route through the
|
||||
// "custom" provider (OpenAI-compatible wire). This honors the user's
|
||||
// intent: passing a custom URL means "use THIS endpoint", not "speak
|
||||
// the Google/Anthropic/etc. wire protocol against this endpoint".
|
||||
//
|
||||
// Any provider prefix on the model is stripped so a model name that
|
||||
// happens to collide with a known provider (e.g. `google/gemma-4-12b`
|
||||
// served by LM Studio) still resolves correctly. If you genuinely need
|
||||
// to point a non-OpenAI wire (Anthropic, Google, ...) at a proxy URL,
|
||||
// use the explicit `custom/<name>` form to opt out of the rewrite by
|
||||
// configuring the proxy as that provider in your config file instead.
|
||||
if viper.GetString("provider-url") != "" && modelFlagChanged {
|
||||
model := viper.GetString("model")
|
||||
if model != "" && !strings.Contains(model, "/") {
|
||||
viper.Set("model", "custom/"+model)
|
||||
if model != "" {
|
||||
name := model
|
||||
if _, after, ok := strings.Cut(model, "/"); ok {
|
||||
name = after
|
||||
}
|
||||
if !strings.HasPrefix(model, "custom/") {
|
||||
viper.Set("model", "custom/"+name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -133,6 +133,15 @@ For self-hosted or proxy endpoints:
|
||||
kit --provider-url "https://my-proxy.example.com/v1" --model openai/gpt-4o
|
||||
```
|
||||
|
||||
When `--provider-url` is set with an explicit `--model`, Kit routes through the
|
||||
`custom` (OpenAI-compatible) wire and strips any provider prefix from the model
|
||||
name. So `openai/gpt-4o`, `google/gemma-4-12b`, and bare `gpt-4o` all resolve
|
||||
to the same endpoint — Kit treats `--provider-url` as authoritative about *where*
|
||||
to send the request, and the model string as just the upstream model id.
|
||||
|
||||
This avoids name collisions when a local server (LM Studio, Ollama, vLLM, ...)
|
||||
happens to expose a model whose name matches a known cloud provider.
|
||||
|
||||
When `--provider-url` is provided without `--model`, Kit automatically defaults to `custom/custom`:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user