Repository navigation
Conversation
…nerator _ollama_stream() is an async def that iterated ollama.Client.chat(stream=True), which returns a synchronous generator: every next() blocked the event loop for the length of that network read, so a streaming Ollama reply froze every other coroutine for the whole response. Await AsyncClient.chat() instead. Both clients are now built from one _ollama_connection_options() helper, so the streaming and non-streaming paths cannot drift onto different endpoints. tests/test_ollama_stream_nonblocking.py drives the stream with fake clients and measures the loop: a ticker coroutine gets 0 turns on the old code and double-digit turns on the new one, while the chunk contract (text in order, empty content dropped) is pinned in the same file. Closes knight22-21#103
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_ollama_stream()awaited nothing: it iteratedollama.Client.chat(stream=True), which returns a synchronous generator, so everynext()blocked the loop for the length of that network read. Theasync defwrapper only made the values awaitable, not the reads.Switched the stream to
ollama.AsyncClient, which returns an async iterator for the same request.Change
devagent/core/llm.py:_ollama_stream()now uses an async client and doesasync for chunk in await client.chat(**kwargs)._ollama_connection_options()helper. They serve the same config from different call sites, so duplicating the host/key logic would have let the streaming path drift onto a different endpoint than the non-streaming one.tests/test_ollama_stream_nonblocking.py(new): fake clients, no Ollama server contacted.The non-streaming paths are untouched —
complete()/complete_with_tools()are sync and are already called throughrun_in_executor, so_ollama_stream()was the only Ollama call running on the loop.Test
test_ollama_stream_leaves_the_event_loop_responsiveconsumes the stream while a ticker coroutine counts its turns; the fake client sleeps between chunks as a network read would.Same test, both directions:
_ollama_stream()AsyncClient)test_ollama_stream_yields_every_non_empty_chunkpins the chunk contract (text in order, empty content dropped) and passes on both trees, so the fix cannot quietly change what callers receive.Local runs
pytest tests/test_ollama_stream_nonblocking.py tests/test_phase27.py tests/test_phase28.py tests/test_phase16.py tests/test_phase23.py -q— 62 passed.mainat5345a78: 7 failed, 835 passed, 1 skipped.The same 7 tests fail on both trees (
test_phase17.py,test_phase38.py) withUnicodeEncodeError: 'gbk' codec can't encode character '\xe2'— a subprocess writing a non-ASCII character under a GBK console on this Windows host, unrelated to this change and left alone.Closes #103