Skip to content

fix(providers): disable AsyncOpenAI keep-alive reuse to avoid SSL UNEXPECTED_EOF on idle-reaping gateways - #205

Closed
raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:pr/fix-provider-keepalive
Closed

fix(providers): disable AsyncOpenAI keep-alive reuse to avoid SSL UNEXPECTED_EOF on idle-reaping gateways#205
raymondginger2018-sudo wants to merge 1 commit into
HKUDS:mainfrom
raymondginger2018-sudo:pr/fix-provider-keepalive

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

This is the remaining, independent half of #198: disable keep-alive connection reuse in OpenAICompatProvider’s AsyncOpenAI client so third-party gateways (e.g. scnet) that reap idle connections don’t hand back a dead connection, which surfaces as SSL: UNEXPECTED_EOF_WHILE_READING.

The GLM-5.2 catalog entry from #198 was already merged upstream as a standalone commit credited to me (329e98f0), so this PR contains only the keep-alive fix.

Changes

  • core/providers/openai_compat.py: pass http_client=httpx.AsyncClient(limits=httpx.Limits(max_keepalive_connections=0)) to AsyncOpenAI (+8 lines). New connection per request (~50–100 ms handshake) is negligible for LLM-second-scale calls.

Tests

  • python -m py_compile core/providers/openai_compat.py
  • module import smoke ✓ (httpx available)
  • full CI pending

Notes

  • Based on current upstream/main (c0a6a3c); single file, no CLAUDE.md / .gitleaksignore changes.

Related: #198

…UNEXPECTED_EOF from gateway idle-connection reaping (scnet)
@Zongwei9888

Copy link
Copy Markdown
Collaborator

Closing for the same reason as the keep-alive half of #198: disabling keep-alive here is hard-coded for every OpenAI-compatible connection (OpenRouter, DeepSeek, Ollama, …), not just the gateway that reaps idle connections, and the custom httpx.AsyncClient is never closed.

The shape that would be accepted is a connection-level option — for example httpKeepAlive: false on ConnectionProfileConfig — read in OpenAICompatProvider.__init__, defaulting to the current behaviour, with the client closed on provider shutdown. A PR in that shape is welcome.

@Zongwei9888 Zongwei9888 closed this Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants