Ollama v1 OpenAI Compat Drops Think Toggle

We recently ran into an issue with the Ollama /v1 endpoint, which is designed to be compatible with OpenAI's API. Our models, particularly those from the Qwen3 family, were returning empty content. This was not the behavior we expected, given the models' capabilities.

We started by examining the requests and responses to identify what was going wrong. We noticed that the /v1 endpoint was silently dropping the think: false toggle for the Qwen3-family models. This toggle is crucial as it controls whether the model spends its entire num_predict budget on internal reasoning or not.

Here's a snippet of the payload we were sending:

{ "model": "qwen3", "messages": [ { "role": "user", "content": "Translate 'hello' to French." } ], "think": false } Enter fullscreen mode Exit fullscreen mode We expected the model to translate "hello" to French without extensive internal reasoning. However, the model was spending its entire num_predict budget on reasoning and returning empty content.

To fix this issue, we switched to the native /api/chat endpoint provided by Ollama. This endpoint allows us to have more control over the model's behavior and ensures that the think toggle is respected.

Here's the payload we used with the /api/chat endpoint:

{ "model": "qwen3", "messages": [ { "role": "user", "content": "Translate 'hello' to French." } ], "think": false } Enter fullscreen mode Exit fullscreen mode By switching to the /api/chat endpoint, we were able to control the model's behavior more effectively and get the expected output.

Switching to the /api/chat endpoint has its own set of tradeoffs. While it gives us more control over the model's behavior, it also requires us to refactor our code to handle the new endpoint. Additionally, the /api/chat endpoint may have different rate limits and pricing structures compared to the /v1 endpoint.

However, the benefits of having more control over the model's behavior and getting the expected output outweigh these tradeoffs for us.

We are currently exploring other endpoints and features provided by Ollama to see if they offer any additional benefits for our use case. We are also working on improving our monitoring and alerting systems to catch such issues earlier.